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sample  of  88  defense  contracts  extracted  from  the  Defense  Acquisition  Executive 
Summary  database.  The  overall  results  of  the  analysis  were  also  tested  to  determine 
which  EAC  methods  performed  the  best  given  such  moderator  variables  as  stage  of 
contract  completion,  system  type,  program  phase,  contract  type,  DoD  service  component, 
and  inflation  effects. 
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Abstract 

Controlling  costs  in  the  acquisition  of  new  defense  systems  is  a  major  challenge  in 
today’s  environment  of  declining  budgets  and  rapidly  changing  technology.  One  of  the 
challenges  faced  by  program  managers  and  cost  analysts  is  selecting  the  most  appropriate 
Estimate  at  Completion  (EAC)  method  for  their  program.  This  study  compares  the 
performance  of  the  popular  index-based  EAC  methods  with  several  newer  nonlinear, 
regression  based  EAC  methods  to  determine  whether  the  complex  nonlinear  methods 
perform  better  than  the  simpler  index-based  methods.  In  addition,  the  sensitivity  of  the 
results  to  stage  of  contract  completion,  system  type,  program  phase,  contract  type. 
Department  of  Defense  service  component,  and  inflation  effects  are  also  investigated. 

Eighty-eight  contracts  were  examined  in  this  study  and  it  was  found  that  overall  the 
index-based  EAC  methods  performed  significantly  better  than  the  nonlinear  regression 
based  methods  as  measured  by  two  criteria,  the  accuracy  and  stability  of  the  EACs.  In 
addition,  the  top  performing  method  overall  was  determined  to  be  the  index-based 
method  using  the  Composite  Index  (0.2SPIcum+0.8CPICUm)-  The  best  performing  method 
was,  however,  sensitive  to  all  of  the  factors  investigated  in  the  sensitivity  analysis. 
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A  COMPARATIVE  STUDY  OF  LINEAR  AND  NONLINEAR  ESTIMATE  AT 


COMPLETION  METHODS 


I.  Introduction 


General  Issue 

Controlling  costs  in  the  acquisition  of  new  weapon  systems  is  a  major  challenge  in 
today’s  environment  of  declining  defense  budgets  and  rapidly  changing  technology.  In 
fact,  research  by  The  RAND  Corporation  found  an  average  of  20  percent  cost  growth 
(excluding  inflation  effects  and  increased  quantity  buys)  in  the  197  defense  programs  it 
studied  (Drezner,  Jarvaise,  Hess,  Hough,  and  Norton,  1993).  A  key  conclusion  of  this 
study  was  “that  cost  estimates  are,  on  average,  systematically  biased  downward,  resulting 
in  cost  growth’’  (Drezner,  et  al.,  1993:49).  The  authors  imply  that  the  methods  used  to 
calculate  the  final  costs  of  defense  acquisition  contracts,  termed  Estimates  at  Completion 
(EACs),  are  not  accurate  and  generally  underestimate  the  true  final  costs. 

Recent  examples  of  failure  to  control  costs  include  the  Navy’s  A- 12  program,  which 
was  canceled  in  January  of  1991  (Morrison,  1991),  and  the  Air  Force  s  C-17  program, 
which  has  faced  constant  criticism  at  the  congressional  level  (Morrocco,  1 993).  One 
reason  Defense  Secretary  Dick  Cheney  canceled  the  Navy  s  A- 12  program  was  because 
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no  one  could  give  him  a  good  estimate  of  the  final  expected  cost  of  the  program 
(Morrison,  1991). 

Background 

Part  of  the  problem  is  that  there  are  numerous  methods  for  calculating  an  EAC. 

These  methods  range  from  simple  index-based  methods  to  sophisticated  statistical 
techniques.  In  Estimate-At-Completion  Research-A  Review  and  Evaluation.  McKinney 
listed  47  EAC  formulas  from  18  sources  (1991).  In  addition  to  the  sources  and  methods 
reviewed  by  McKinney,  others  have  proposed  sophisticated  nonlinear  regression-based 
methods  involving  the  Rayliegh  probability  distribution  (Watkins,  1982;  Abernathy, 
1984).  the  Rayliegh  probability  distribution  coupled  with  Multiple  Model  Adaptive 
Estimation  (MMAE)  (Gallagher  and  Lee,  1 995),  and  a  modified  Beta  distribution 
(Whitlock,  1982).  With  so  many  methods  available,  choosing  the  best  one  is  difficult. 

A  number  of  studies  have  compared  EAC  methods  using  actual  defense  program  cost 
data  (Heydinger,  1977;  Land  and  Preston,  1980;  Covach,  Haydon,  and  Riether,  1981; 
Bright  and  Howard,  1981;  Riedel  and  Chance,  1989;  Terry  and  Vanderburgh,  1993). 
Unfortunately,  much  of  the  comparative  research  has  focused  on  the  simpler  index-based 
methods.  The  studies  that  did  include  other  methods,  such  as  regression-based  methods, 
have  produced  inconclusive  results  (Heydinger,  1977;  Land  and  Preston,  1980;  Covach, 
et  aJ. ,  1981;  Bright  and  Howard,  1981).  In  addition,  none  of  these  comparative  studies 
included  any  of  the  more  advanced  regression-based  methods  involving  the  Rayleigh 
distribution  or  the  modified  Beta  distribution.  In  “A  Review  of  Estimate  at  Completion 
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Research,”  Christensen,  Antolini,  and  McKinney  conclude  that  there  has  been  no  real 
validation  of  the  more  complex  EAC  methods,  and  that  “the  accuracy  of  regression-based 
models  over  index-based  formulas  has  not  been  established”  (1992:220). 

Research  Problem 

The  primary  objective  of  this  research  is  to  determine  if  the  complex  nonlinear 
regression-based  EAC  methods  perform  better  than  the  simpler  index-based  EAC 
methods.  In  this  study,  the  performance  of  the  EAC  methods  was  compared  using  a 
combined  measure  that  ranked  performance  against  two  criteria,  the  accuracy  and  stability 
of  the  estimates  generated  by  the  method.  The  method  that  was  the  most  accurate  and 
stable  overall  was  classed  as  the  best  performing  method.  A  more  detailed  description  of 
this  combined  measure  is  reserved  for  Chapter  III. 

Specific  Problem  Statement 

Do  complex  nonlinear  regression-based  EAC  methods  perform  better  than  simple 
index-based  EAC  methods?  The  primary  hypothesis  to  be  tested  is: 

EACs  calculated  using  complex  nonlinear  regression-based  methods  are  not  more 

accurate  or  stable  than  EACs  calculated  using  simple  index-based  methods. 

This  hypothesis  was  tested  using  data  extracted  from  the  Defense  Acquisition 
Executive  Summary  (DAES)  database.  This  database  is  a  collection  of  cost  performance 
data  from  541  completed  or  near  completed  defense  contracts  covering  the  period  from 
June  1970  to  October  1992.  The  database  includes  both  development  and  production 
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contracts  from  all  three  services  and  a  variety  of  different  system  types.  The  data  were 
screened  to  eliminate  contracts  with  incomplete  or  insufficient  data  before  performing  the 
analysis. 

Additional  Research  Objectives 

Before  addressing  the  primary  research  objective,  two  additional  objectives  needed  to 
be  accomplished.  First,  the  impact  of  cost  growth  on  DoD  acquisition  programs  had  to  be 
established  to  illustrate  the  need  for  better  performing  EAC  methods.  And  second,  it  was 
necessary  to  review  previous  research  to  determine  which  EAC  methods  would  be 
included  in  this  study.  Both  of  these  objectives  were  accomplished  through  a  review  of 
relevant  literature. 

In  addition  to  answering  the  primary  research  question,  it  was  also  desirable  to 
determine  which  EAC  methods  performed  the  best  given  such  moderator  variables  as 
stage  of  contract  completion,  system  type,  program  phase,  contract  type,  DoD  service 
component,  and  inflation  effects. 

Scope  and  Limitations 

The  data  used  for  analysis  in  this  study  were  limited  to  contracts  extracted  from  the 
DAES  database.  No  attempt  was  made  to  locate  other  data  to  include  in  the  study  due  to 
time  constraints.  Due  the  nature  of  the  database,  it  is  considered  to  provide  a  sufficiently 
representative  sample  of  all  DoD  contracts. 

This  study  was  limited  to  identifying  the  best  performing  EAC  methods  overall,  and 
subject  to  the  moderator  variables  previously  discussed.  No  attempt  was  made  to  conduct 
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further  analysis  to  determine  the  underlying  factors  that  influenced  the  performance  of  a 
given  method.  This  additional  analysis  would  have  required  much  more  time  than  was 
available  to  conduct  this  research. 

The  following  chapters  provide  a  review  of  previous  literature  relevant  to  this 
comparative  EAC  study,  the  methodology  used  in  this  study,  an  analysis  of  the  results, 
conclusions,  and  recommendations  for  further  research. 
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II.  Literature  Review 


Introduction 

Controlling  costs  in  the  acquisition  of  new  weapon  systems  is  a  major  challenge  in 
today’s  environment  of  declining  defense  budgets  and  rapidly  changing  technology.  One 
major  aspect  of  controlling  costs  is  accurately  estimating  the  final  cost  of  a  defense 
acquisition  contract,  termed  Estimate  at  Completion  (EAC).  There  are  many  methods  of 
calculating  an  EAC,  ranging  from  very  simple  index-based  methods  to  sophisticated 
regression-based  and  statistical  techniques.  With  such  a  wide  variety  of  methods, 
choosing  the  method  that  provides  the  most  accurate  forecast  is  not  a  simple  matter.  The 
objective  of  this  thesis  is  to  determine  whether  or  not  complex  nonlinear  regression-based 
EAC  methods  perform  better  than  the  simpler  index-based  methods  based  on  two  criteria, 

accuracy  and  stability.  This  literature  review  will  focus  on  related  EAC  research  and 
findings. 

This  chapter  is  divided  into  five  sections.  The  first  section  addresses  cost  growth  in 
DoD  acquisition  programs,  stressing  the  need  for  accurate  EACs.  The  second  section 
describes  the  two  major  classes  of  EAC  methods  relevant  to  this  study,  and  also  includes 
a  review  of  the  EAC  methods  used  in  DoD’s  Performance  Analyzer  software  package  in 
order  to  demonstrate  the  popularity  of  these  methods.  The  third  section  is  a  review  of 
comparative  EAC  studies.  The  fourth  section  introduces  two  of  the  more  significant 
nonlinear  models  that  have  been  proposed,  as  well  as  methods  of  calculating  EACs  using 
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these  models,  and  the  fifth  section  provides  concluding  remarks  and  discusses  the 
implications  and  potential  outcomes  of  this  research  effort. 

Cost  Growth  in  DoD  Acquisition  Programs 

Cost  growth,  in  its  simplest  form,  can  be  defined  as  any  increase  over  the  original 
cost  estimate  for  a  program.  Although  some  evidence  suggests  that  cost  growth  has 
decreased  since  the  1950s  (Biery,  1984),  a  1993  study  by  RAND  corporation  contends 
that  it  has  remained  constant  (Drezner,  et  al.,  1993).  In  fact,  the  RAND  research  found  an 
average  growth  rate  of  20  percent  for  the  197  programs  studied  (Drezner,  et  al.,  1993). 

To  illustrate  the  significance  of  a  20  percent  growth,  consider  the  1993  estimate  of  41 
billion  dollars  for  the  Air  Force’s  C-l  7  program  (Morrocco,  1 993).  A  20  percent  increase 
is  8.2  billion  dollars.  Another  alarming  observation  reported  by  Christensen,  is  that 
recovery  from  a  cost  overrun  (i.e.,  cost  growth)  on  a  defense  contract  is  highly  unlikely, 
and  that  cost  overruns  tend  to  worsen  as  a  contract  progresses  to  completion  (1 993). 

A  major  emphasis  in  DoD  acquisition  policy  reforms  since  the  early  1960s  has  been 
on  ways  to  control  cost  growth  (Gates,  1989).  One  of  the  key  developments  that  evolved 
from  these  reforms  is  our  current  Cost/Schedule  Control  Systems  Criteria  (C/SCSC). 

These  criteria  were  developed  in  the  mid  1 960s  as  the  result  of  an  Air  Force  study  group 
known  as  the  Cost/Schedule  Planning  and  Control  Specification  Group  (Fleming,  1992). 
C/SCSC  are  a  set  of  35  criteria,  or  standards,  which  are  designed  to: 

Provide  and  adequate  basis  for  responsible  decision  making  by  both 
contractor  management  and  DoD  Component  personnel  by  requiring  that 
contractors’  internal  management  control  systems  produce  data  that: 
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(a)  Indicate  work  progress; 

(b)  Properly  relate  cost,  schedule,  and  technical  accomplishment; 

(c)  Are  valid  and  timely,  and  able  to  be  audited;  and 

(d)  Provide  DoD  Component  managers  with  information  at  a  practical 
level  of  summarization  (Department  of  Defense,  1 99 1:11  -B- 1  -  ll-B-2) 

In  other  words,  these  criteria  are  designed  to  ensure  that  cost,  schedule,  and  performance 
data  provided  by  the  contractor  can  be  used  to  accurately  monitor  the  progress  of  the 
contract  and  ultimately  to  forecast  future  contract  performance.  An  EAC  is  simply  a 
forecast  of  future  cost  performance,  specifically,  the  Cost  at  Completion  (CAC). 

The  importance  of  accurately  forecasting  contract  completion  costs  cannot  be 
overstated.  In  today’s  post-cold  war  environment  defense  budgets  are  shrinking,  resulting 
in  competition  and  dispute  within  DoD  for  the  available  funds  (Jones,  1993).  “Without 
more  realistic  estimates,  senior  management  may  be  lulled  into  a  false  sense  of  security 
about  their  programs  and  fail  to  take  appropriate  action  to  correct  problems” 

(Christensen,  1993).  In  other  words,  senior  managers  need  accurate  EACs  to  identify 
problems  early  so  the  problems  can  be  corrected,  or  if  they  cannot  be  corrected,  the 
program  can  be  canceled  before  a  significant  investment  is  made. 

Methods  of  Calculating  EACs 

There  are  many  methods  of  calculating  an  EAC.  The  two  major  classes  of  EAC 
methods  relevant  to  this  study,  index-based  EAC  formulas  and  regression-based  EAC 
methods,  are  described  in  this  section.  The  section  concludes  with  a  review  of  the  EAC 
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methods  included  in  the  DoD’s  Performance  Analyzer  (Cost  Management  Systems,  Inc., 
1994)  software  package. 

Index-Based  EAC  Methods.  The  index-based  methods  are  basically  an  algebraic 
formula  that  uses  an  index  to  calculate  the  EAC.  An  index  is  a  measure  of  the  level  of 
performance  attained  in  completing  the  work  on  a  contract  up  to  the  current  time.  The 
general  form  for  the  index-based  EAC  formula  is  as  follows: 


EAC  =  ACWPcum 


+  (BACIBCWP_) 
Index 


0) 


where  ACWPcum  is  the  cumulative  Actual  Cost  of  Work  Performed  (i.e.,  the  cost 
incurred  to  date  in  performing  the  contract),  BAC  -  BCWP,.„m  is  the  Budget  at 
Completion  minus  the  cumulative  Budgeted  Cost  of  Work  Performed  (i.e.  total  budget 
for  the  contract  minus  the  budgeted  cost  of  the  work  completed  to  date),  and  Index  is 
one  of  several  performance  indices  as  defined  below.  For  the  reader  unfamiliar  with 
these  terms,  a  glossary  is  provided  in  Appendix  A. 

The  four  major  indices  include  the  Cost  Performance  Index  (CPI),  Schedule 
Performance  Index  (SPI),  Schedule  Cost  Index  (SCI),  and  the  Composite  Index.  The  CPI 
can  be  calculated  as  a  cumulative  number 


CPI 


cum 


BCWPci 

ACWP, 


or,  in  two  different  forms  as  a  moving  average 


(2) 
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CPIn  = 


XBCWPn 

5>CWP„ 


CPI  n  =  £CPI" 


n 


(3) 

(4) 


where  n  represents  the  number  of  months  of  data  to  be  summed,  beginning  with  the 
most  recent  month.  The  use  of  mathematical  symbology  to  distinguish  between  these  two 
forms  of  the  CPI  was  inconsistent  in  the  literature  reviewed.  Because  the  standard  use  of 
the  (bar)  symbol  in  math  and  science  is  to  denote  an  average,  I  have  chosen  to  use  it  in 
the  same  manner.  Therefore,  CPI3  denotes  the  sum  of  the  three  most  recent  monthly 

BC  WPs  divided  by  the  sum  of  the  three  most  recent  monthly  ACWPs,  and  CPI  6  denotes 
the  average  of  the  six  most  recent  monthly  CPIs.  Note  that  in  this  study,  CPIm  will  be 
used  to  refer  to  the  current  month’s  CPI  rather  than  CPI, . 

The  SPI  can  be  calculated  as  a  cumulative  number 


SPI  -  BCWPcun, 


BCWS„ 


or,  in  two  different  forms  as  a  moving  average 


Vbcwp 

SPIn=  ^ 


2>cwsn 


—  Yspi 

spi  n  =  — — i 


n 


(5) 

(6) 

(7) 
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where  n  again  represents  the  number  of  months  of  data  to  be  summed,  beginning  with 
the  most  recent  month.  Like  the  CPI3,  SPI3  denotes  the  sum  of  the  three  most  recent 
monthly  BCWPs  divided  by  the  sum  of  the  three  most  recent  monthly  BCWSs,  while 
SPI  6  denotes  the  average  of  the  six  most  recent  monthly  SPIs. 

The  SCI  is  the  product  of  the  SPI  and  CPI  and  is  calculated  as  follows: 

SCI  =  SPIxCPI  (8 

The  SCI  can  be  calculated  using  either  the  monthly,  moving  average,  or  cumulative  SPI 
and  CPI.  For  example,  SCI3  would  be  calculated  using  SPI3  and  CPI3,  and  SCIcum  would 
be  calculated  using  SPICUI71  and  CPIcum.  In  addition,  modified  versions  of  the  SCI  can  be 
calculated  using  various  combinations  of  the  monthly,  moving  average,  and  cumulative 
SPI  and  CPI.  An  example  of  this  is  the  index-based  EAC  formula  in  Performance 
Analyzer  that  uses  SPIcumxCPl6  (Cost  Management  Systems,  Inc.,  1994). 

The  fourth  index  is  the  Composite  Index,  which  is  the  weighted  sum  of  the  SPI  and 
CPI,  is  calculated  as  follows: 

Composite  Index  =  W)xSPI+W2xCPI  (9} 

where  Wi  and  W2  are  the  weights  assigned  by  the  analyst  to  the  SPI  and  CPI 
respectively.  These  weights  can  take  on  any  value  between  zero  and  one,  and  typically 
sum  to  unity. 

Note  that  in  the  convention  used  in  this  study  an  index  value  greater  than  one 
indicates  good  performance  (i.e.  under  budget  or  ahead  of  schedule).  For  example: 
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BCWPcum  =  $10,000 
ACWPcum  =  $9,000 


CPI  =  BCWPcu, 
V~1  1cum 


ACWP„ 


10,000 


9,000 


In  this  case  the  actual  cost  to  complete  the  work  ($9,000)  was  less  than  originally 
budgeted  ($1 0,000)  resulting  in  a  CPI  greater  than  one.  This  would  generally  be  seen  as  a 
positive  indicator.  Conversely,  an  index  value  less  than  one  would  indicate  poor 
performance  (i.e.,  over  budget  or  behind  schedule). 

Regression-Based  EAC  Methods.  These  are  methods  used  to  calculate  the  EAC  by 
regressing  a  dependent  variable,  typically  ACWP,  against  an  independent  variable  such  as 
BCWP  or  time.  The  resulting  regression  equation  is  then  used  to  calculate  the  EAC. 
Regression  methods  can  either  be  linear  (e.g.,  ACWP=AxBCWP+B  ,  where  A  and  B 
are  the  coefficients  of  regression),  or  nonlinear  (e.g.,  ACWP=AxBCWPb  ) 

Review  of  Performance  Analyzer  EAC  Methods.  Performance  Analyzer  k 
DoD  s  standard  software  package  for  analyzing  C/SCSC  data.  One  of  the  functions  of 
the  software  allows  the  user  to  generate  independent  program  office  EACs  based  on  the 
data  submitted  by  the  contractor  in  the  monthly  Cost  Performance  Reports  (CPRs)  or 
Cost/Schedule  Status  Reports  (C/SSRs).  There  are  12  different  methods  the  user  can 
choose  from  to  generate  an  EAC,  including  eight  index-based  methods  (CPI3,  CPI6, 

CPIcum,  CPIm,  SCIcum,  Composite  Index,  SPIcumxCPI6,  and  a  user  determined  performance 
factor),  one  linear  regression  technique  where  ACWP  is  regressed  as  a  function  of 
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BCWP,  and  a  complex  linear  formula  based  on  historical  data  from  NAVSEA  contracts. 
The  two  remaining  methods  allow  the  contract  BAC  or  the  contractor’s  LRE  to  be  used  as 
the  EAC  (Cost  Management  Systems,  Inc.,  1994).  The  heavy  emphasis  on  the  index- 
based  methods  demonstrates  the  popularity  of  these  methods  warranting  their  inclusion  in 
a  comparative  study  of  EAC  methods.  It  should  also  be  noted  that  there  no  EAC  methods 
involving  nonlinear  models  in  Performance  Analyzer. 

Comparative  EAC  Studies 

The  body  of  EAC  research  reviewed  can  be  separated  into  two  general  categories, 
research  which  is  aimed  at  validating  or  comparing  previously  developed  methods,  and 
research  which  is  aimed  at  developing  new  EAC  methods.  This  section  focuses  on 
several  comparative  studies  that  have  been  done,  as  well  as  two  recent  studies  aimed  at 
validating  several  index-based  methods.  The  following  section  introduces  two  of  the 
more  significant  nonlinear  models  that  have  been  proposed  for  calculating  EACs.  Table 
1  provides  a  summary  of  the  comparative  EAC  studies  that  were  reviewed. 
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TABLE  1 


SUMMARY  OF  COMPARATIVE  EAC  STUDIES 


Author 

(Year) 

DoD  Agency,  # 
and  Type  of 
Contracts 

EAC  Methods 

Evaluation  Criteria 

Significant  Results 

Heydinger 

(1977) 

Air  Force, 

1  development 

Index-based 

CPI™  CPI  3, 
CPIcum,  CPI3 
Reeression 

1  linear 

2  nonlinear1 

Consistently  closest  to  final 
cost 

Nonlinear  regression 
method  using  modified 
Erlang  equation  most 
accurate 

Land  and 

Preston 

(1980) 

Air  Force, 

20  aircraft 
programs 

Index-based 

CPIm,  CPI  3, 
CPICUrm  CPI3 
Regression 

1  nonlinear2 

Mean  absolute  percentage 
error  (MAPE) 

Nonlinear  regression 
methods  do  not  produce 
more  accurate  EACs; 

CPIcum  had  lowest  MAPE 

Covach.  et  al. 
(1981) 

Navy, 

5  development 

1  production 

Index-based 

CP|m-  CPU 
CPU  CPI  ,2- 

CPIcum^  SCIcum, 
CPI3,  CP16 
Reeression 

2  linear 

9  nonlinear3 

#  of  months  EAC  within 

10%  of  CAC; 

#  of  months  EAC  closer  to 
CAC  than  BAC; 

#  of  months  EAC  closer  to 
CAC  than  contractor's  LRE 

Best  performing  indices 
dependent  on  stage  of 
contract  completion; 

No  comparison  of  index- 
based  and  regression 
methods; 

No  single  regression 
method  demonstrated  clear 
superiority 

Bright  and 

Howard 

(1981) 

Army, 

1 1  development 

Index-based 

CPU,  CPU, 

CPI  ,*  CPIcum, 
50/504,  75/254, 

SPIcumxCPI6 

Regression 

1  linear 

1  nonlinear5 

Average  percent  error 
plotted  as  a  function  of 
time,  plots  of  each  EAC 
method  compared  to 
determine  which  was 
closest  to  final  cost  at  each 
stage  of  completion 

Most  accurate  method 
dependent  on  stage  of 
contract  completion; 
Index-based  method  using 

CPI  ^xSPIcum  performed 
best  overall 

Riedel  and 

Chance 

(1989) 

Air  Force, 

16  development 
48  production 

Index-based 

CPI  SCI 

Acum>  1curr»' 

20/804,  cpim, 

CPI3,  PC6 

MAPE 

Accuracy  of  EAC  methods 
dependent  on  stage  of 
completion,  type  of 
weapons  system,  and 
program  phase 

Notes: 

1  ACWP=AxBCWPb  and  ACWP=AxXBxecx,  where  X  =  months 

2  ACWP=AxBCWPb 

3  Y=A+BX;  Y=AXb;  Y=AeBX;  and.  Y=A+BlogeX  (where  ACWP  (Y)  was  regressed  against  BCWP  (X),  CPI  (Y) 
was  regressed  against  time  (X),  and  both  ACWP  (Y)  and  BCWP  (Y)  were  regressed  separately  against  time  (X)) 

4  W1/W2:  Composite  Index  =  W1xSPIculll+W2xCPIclini  (e.g.,  50/50  is  .50SPIcum+.50CPIcum) 

5  Y=AXb  where  CPI  (Y)  was  regressed  against  time  (X) 

^TC^ComgositeaIndex^(PC2CPlg^y^PC|SPIajg whe^^^^rcentComgl«e=t^^^==a=^=== 
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Hevdinger.  The  earliest  comparative  EAC  study  reviewed  was  the  Space  and 
Missile  Systems  Organization  (SAMSO)  study  done  by  Heydinger  in  1977.  This  study 
tested  four  index-based  methods  and  three  regression  models  using  42  months  of  data 
from  a  single  Air  Force  development  contract.  The  index-based  methods  included  CPIm, 

CPI  3,  CPIcum,  and  CPI3.  The  three  regression  models  included  one  where  ACWP  was 
regressed  against  BCWP  in  the  nonlinear  form  ACWP=AxBCWPb  ,  and  two  where 
BC  WP  and  ACWP  were  regressed  against  time,  first  as  linear  functions  of  time,  and  then 
using  the  nonlinear  form  AxXBxecx  (a  modified  Erlang  equation),  where  X=month  . 
Using  the  evaluation  criterion  of  “consistently  closest  to  final  cost”  as  his  measure  of 
comparison,  Heydinger  found  that  the  nonlinear  regression  model  using  the  modified 
Erlang  equation  was  the  most  accurate  over  the  life  of  the  program.  Of  the  remaining 
methods,  Heydinger  found  the  regression  model  where  ACWP  was  regressed  against 
BCWP  in  the  nonlinear  form  ACWP=AxBCWPb  ,  to  be  a  good  estimator  during  the 
early  (3rd  to  7th  months)  and  late  (27th  to  42nd  months)  stages  of  the  contract,  and  the 
index-based  method  using  CPI  3  to  be  a  good  estimator  during  the  middle  stage  (8th  to 
26th  months). 

Land  and  Preston.  A  second  study  done  by  Land  and  Preston  in  1 980,  using  data 
from  20  Air  Force  aircraft  programs,  tested  the  same  four  index-based  methods  as  the 
Heydinger  study  (CPIm,  CPI  3,  CPICUm,  and  CPI3)  and  the  regression  model  where  ACWP 
was  regressed  against  BCWP  in  the  nonlinear  form  ACWP=AxBCWPb  .  The 
regression  was  run  with  the  regression  coefficients  A  and  B  unconstrained  (i.e.,  the  values 
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of  A  and  B  were  determined  by  the  regression)  and  again  with  the  coefficient  B 
constrained  (i.e.,  the  value  of  B  was  fixed  and  only  the  value  of  A  was  determined  by  the 
regression).  As  a  measure  of  EAC  accuracy,  Land  and  Preston  used  the  Mean  Absolute 
Percentage  Error  (MAPE),  defined  as  “the  estimated  value  minus  the  actual  final  cost  at 
completion  divided  by  the  actual  final  cost”  (Land  and  Preston,  1980:50).  In  general,  the 
study  found  that  the  nonlinear  regression-based  methods  did  not  result  in  more  accurate 
EACs,  and  the  method  with  the  lowest  MAPE  was  the  index-based  method  using  CPIcum. 

Covach.  el  al.  A  1 98 1  ManTech  International  Corporation  study  by  Covach,  et  al. , 
tested  1 2  index-based  methods  and  1 2  regression  equations  using  data  from  six  Navy 
contracts  (five  research  and  development  (R&D)  and  one  production).  The  index-based 

methods  included  CPIm,  CPI  3,  CPI  6,  CPI  12,  CPICUm,  SCICUm,  CPI3,  and  CPI6.  Four  of 
the  1 2  index-based  methods  divided  BAC  by  CPI  3,  SPIcum,  CPI3,  and  CPI6,  which  “is  an 
incorrect  algebraic  simplification  of  the  basic  EAC  formula,”  Equation  (1)  (Christensen, 
et  al.,  1992:217).  The  12  regression-based  methods  used  the  forms  Y=A+BX  ,  Y=AXB  , 
Y=AeBX  ,  and  Y=A+BlogeX  ,  and  regressed  ACWP  against  BCWP,  CPI  against  time, 
and  both  ACWP  and  BCWP  separately  against  time  using  each  of  these  forms. 
Unfortunately,  no  comparisons  were  made  between  the  index -based  methods  and  the 
regression-based  methods. 

In  this  study,  three  different  measures  were  used  to  compare  the  various  methods. 

The  first  measure  was  the  number  of  months  that  the  EAC  was  within  1 0%  of  the  actual 
CAC.  The  second  was  the  number  of  months  that  the  EAC  was  closer  to  the  actual  CAC 
than  the  BAC.  The  third  was  the  number  of  months  that  the  EAC  was  closer  to  the  actual 
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CAC  than  the  contractor’s  Latest  Revised  Estimate  (LRE).  Success  was  defined  for  each 
measure  as  the  EAC  being  more  accurate  greater  than  or  equal  to  75%  of  the  time.  Failure 
was  defined  as  the  EAC  being  more  accurate  less  than  50%  of  the  time  and  the  range 
from  50%  to  74%  was  defined  as  indifferent.  An  overall  score  for  each  method  was 
calculated  by  awarding  + 1  to  each  method  for  each  success,  0  (zero)  for  each  indifferent 
result,  and  -1  for  each  failure.  In  addition  to  an  overall  evaluation,  the  methods  were 
evaluated  at  various  stages  of  contract  completion,  as  defined  in  Table  2. 

The  best  performing  index-based  methods  overall  were  CPI6  and  CPI  n-  The  results 
for  the  index-based  methods  by  stage  of  contract  completion  are  shown  in  Table  2.  It  is 
significant  to  note  that  the  best  performing  indices  differed  depending  on  the  stage  of 
contract  completion,  with  only  the  CPI3  index  performing  well  in  all  three  stages. 

Overall,  the  results  from  comparing  the  regression  equations  were  inconclusive.  No 
single  method  demonstrated  clear  superiority;  however,  once  a  given  method  started  to 
perform  well,  it  would  typically  remain  the  best  method  over  the  life  of  a  contract 
(Covach,  et  ai,  1981). 


TABLE  2 

RESULTS  OF  EAC  COMPARATIVE  STUDY  -  COVACH,  et  ai,  1981 

(Christensen,  et  al.,  1992) 


Contract  Completion  Stage  Best  Performing  EAC  Methods 

Early  (0%  to  40%  Complete)  CPI3,  CPIcum,  SCICUm 

Middle  (20%  to  80%  Complete)  CPI3,  CPICUm,  CPI  6,  SCIcum 
Late  (60%  to  100%  Complete)  CPI3,  CPI6,  CPI  n 
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Bright  and  Howard.  Another  1 98 1  study,  done  by  Bright  and  Howard  from  the 


Army  Missile  Command,  confirmed  the  sensitivity  of  EAC  accuracy  to  contract 
completion  stage.  This  study  examined  eight  index-based  methods  and  two  regression- 
based  methods  using  data  from  1 1  Army  R&D  contracts.  The  index-based  methods 
included  CPI  3,  CPI  6,  CPI  l2,  CPIcum,  0.5SPIcum+0.5CPIcUm,  0.75SPIcum+0.25CPICUm, 
SPIcum,  and  SPIcumx  CPI  6  (a  modified  version  of  the  SCI).  The  regression-based  methods 
included  one  linear  model  of  the  form  Y=A+BX  and  one  nonlinear  model  of  the  form 
Y=AXb  .  In  both  cases,  the  CPI  was  regressed  against  time  and  projected  to  obtain  a  CPI 
at  completion,  which  was  then  used  in  the  general  EAC  formula,  given  in  Equation  (1)  of 
this  paper,  to  arrive  at  the  EAC.  The  various  methods  were  compared  by  plotting  the 
average  percent  error  (relative  to  final  costs)  of  each  estimating  technique  as  a  function  of 
time,  and  then  comparing  these  plots  for  each  method  to  determine  which  was  the  most 
accurate  (closest  to  final  cost)  at  each  stage  of  contract  completion.  Summarized  results 
of  this  study  are  presented  in  Table  3.  Again,  it  is  significant  to  note  that  the  best 
performing  EAC  methods  differ  depending  on  the  stage  of  contract  completion.  In  this 
study,  however,  the  most  consistent  method  was  found  to  be  the  index-based  method 
using  SPICUmx  CPI  6  (Bright  and  Howard,  1981). 
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TABLE  3 


RESULTS  OF  EAC  COMPARATIVE  STUDY  -  BRIGHT  AND  HOWARD,  1981 

(Christensen,  etal.,  1992) 


Early  (0%  to  30%  Complete)  Regression,  Composite,  SPICUm,  SPICumx  CPI  6 

Middle  (31%  to  80%  Complete)  CPI  3,  CPI  6,  CPL12,  SPUm*  CPI  6 
Late  (8 1  %  to  1 00%  Complete)  CPICUm,  SPIcum*  CPI  6 


and  Chance.  A  1989  study  done  at  the  Aeronautical  Systems  Division,  by 
Riedel  and  Chance,  further  confirmed  the  sensitivity  of  EAC  accuracy  to  contract 
completion  stage.  In  addition,  to  contract  completion  stage,  Riedel  and  Chance 
investigated  sensitivity  to  the  type  of  system  (aircraft,  engine,  or  avionics)  and  program 
phase  (development  or  production).  This  study  compared  six  index-based  methods 
(CPU„,  SCIcum,  0.2SPW0.8CPUm,  CPIm,  CPI,,  and  (PC)CPlc„+(l-PC)SPI„n where 
PC  is  the  contract  Percent  Complete)  using  data  from  64  AirTorce  contracts  (16 
development  and  48  production).  As  a  measure  of  EAC  accuracy  the  authors  used  the 
MAPE,  with  the  EAC  method  having  the  lowest  MAPE  being  classed  as  the  most 
accurate.  The  results  of  this  study,  shown  in  Table  4,  clearly  demonstrate  that  the 
accuracy  of  a  given  EAC  method  is  sensitive  to  the  type  of  system  and  the  phase  of  the 
program  in  addition  to  being  sensitive  to  contract  completion  stage. 
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TABLE  4 


RESULTS  OF  EAC  COMPARATIVE  STUDY  -  RIEDEL  AND  CHANCE,  1989 

(Christensen,  etal,  1992) 


Contract  Completion  Stage 


System 

Type 

Program 

Phase 

Number 

of 

Contracts 

25% 

50% 

75% 

100% 

Overall 

Aircraft 

Development 

7 

SCICum 

CPI3 

cpi3 

20/801 

SCIcum 

Aircraft 

Production 

23 

SCIcum 

CPI3 

SCIcum 

CPIcum 

SCIcum 

Avionics 

Development 

5 

SCIcum 

CPI3 

CPI3 

CPIcum 

CPI3 

Avionics 

Production 

16 

20/80 

SCIcum 

20/80 

SCIcum 

20/80 

Engine 

Development 

4 

CPIm 

SCIcum 

CPI3 

CPI3 

CPI3 

Engine 

Production 

9 

PC2 

CPIcum 

SCIcum 

PC 

CPIcum 

Notes: 

1  20/80:  Composite  Index  =  0.2SPIcum+0.8CPICUm 

2  PC:  Composite  Index  =  (PC)CPICum+(l-PC)SPICum  where  PC  =  Percent  Complete 


The  results  of  the  comparative  EAC  studies  discussed  above  are  significant  in  three 
ways.  First,  they  indicate  that  no  single  method  will  always  produce  the  most  accurate 
EAC.  Second,  these  results  establish  the  importance  of  including  additional  analysis  in  a 
comparative  EAC  study  to  determine  the  sensitivity  of  the  results  to  such  factors  as 
program  phase,  system  type,  and  contract  completion  stage.  Third,  with  the  exception  of 
the  Heydinger  study  (1977),  the  nonlinear  EAC  methods  were  not  superior. 

Two  recent  studies,  although  not  comparative  EAC  studies,  provide  additional 
insight  into  the  overall  performance  of  index-based  EAC  formulas.  In  the  first  study, 
Christensen  and  Heise  investigated  the  stability  of  the  CPI  and  found  that  CPIcum  was  a 
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very  stable  indicator  (within  0.2  of  its  final  value  as  early  as  the  20  percent  completion 
point).  Based  on  this  they  concluded  that  any  EACs  lower  than  the  EAC  calculated  using 
CPIcum  were  questionable  and  should  require  strong  justification  (Christensen  and  Heise, 

1993). 

The  second  study,  done  by  Terry  and  Vanderburgh,  tested  the  widely  held  belief  that 
the  CPI-based  EAC  is  a  floor  and  the  SCI-based  EAC  a  ceiling  for  the  final  contract  Cost 
at  Completion  (CAC).  The  authors  found  that  the  CPI-based  EAC  was  the  floor  and  the 
SCI-based  EAC  the  ceiling  for  the  range  of  EACs  calculated;  however,  the  overall  range 
of  EACs  tended  to  underestimate  the  actual  CAC  by  approximately  five  percent  (Terry 
and  Vanderburgh,  1993).  This  range  of  EACs  was  calculated  using  nine  indices  which 
included  CPICUm,  CPU,  CPI3,  SPIcum,  SPI6,  SP13,  SCICUm,  SCI6,  and  SCI3.  These  two 
studies  provide  further  justification  for  including  the  CPI-  and  SCI-based  EAC  formulas 

in  a  comparative  study. 

Based  on  this  review  of  comparative  EAC  studies  it  is  apparent  that  attempts  to 
validate  or  compare  regression-based  methods  have  been  limited,  and  the  results  have 
been  inconclusive.  While  some  of  these  results  indicate  that  regression-based  methods 
can  produce  accurate  EACs  (Heydinger,  1977;  Covach,  a  ai,  1981;  Bright  and  Howard, 
1981),  others  completely  discount  the  use  of  these  methods  (Land  and  Preston,  1980). 
One  common  factor  among  the  regression-based  methods  examined  in  these  studies  is 
that  the  selection  of  a  particular  regression  model,  whether  linear  or  nonlinear,  did  not 
appear  to  be  based  on  any  sound  scientific  evidence.  The  use  of  the  nonlinear  models 
was  an  attempt  to  approximate  the  nonlinear  nature  of  cost  expenditures  on  DoD 
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contracts;  however,  the  models  do  not  appear  to  have  been  tested  to  determine  just  how 
accurately  they  duplicated  this  nonlinear  trend.  The  next  section  describes  two  of  the 
more  popular  nonlinear  models  that  were  found  in  this  review. 

Nonlinear  EAC  Methods 

The  use  of  nonlinear  models  to  approximate  cost  expenditures  and  manpower  usage 
on  R&D  projects  is  clearly  not  a  new  idea.  In  fact,  Norden  cites  research  studies  from  the 
early  1 960s  that  were  successful  in  describing  what  he  calls  the  “regular  patterns  of 
manpower  buildup  and  phase-out  in  complex  projects”  using  several  different 
“mathematical  functions,  generally  in  the  family  of  exponential,  gamma,  beta,  or  logistic 
curves”  (1980:218).  While  Norden’s  focus,  as  well  as  the  focus  of  the  other  nonlinear 
research  reviewed,  was  on  R&D  projects,  no  evidence  was  presented  to  suggest  that  these 
nonlinear  models  did  not  apply  to  production  efforts  as  well.  In  fact,  a  study  by  Knepp 
and  Stroble  found  that  production  efforts  also  displayed  nonlinear  cumulative  cost  curves 
(1994). 

The  two  major  nonlinear  models  discovered  in  this  review  included  the  Rayleigh 
distribution  and  a  modified  Beta  distribution.  This  section  will  review  both,  in  addition 
to  describing  several  methods  of  calculating  EACs  using  these  nonlinear  models. 

Rayleigh  Distribution  EAC  Methods.  Several  studies  have  investigated  the 
application  of  the  Rayleigh  distribution  to  cost  estimation  (Abemethy,  1 984;  Gallagher 
and  Lee,  1995;  Watkins,  1982).  All  three  studies  found  that  ACWP  on  defense 
acquisition  contracts  (the  focus  of  these  studies  was  on  research  and  development 
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contracts)  could  be  modeled  accurately  using  the  Rayleigh  distribution.  The  results  of 
these  studies  are  summarized  in  Table  5. 

TABLE  5 

SUMMARY  OF  RAYLEIGH  DISTRIBUTION  STUDIES 


DoD  Agency 


Author 

(Year) 

(#  and  Type 
of  Contracts) 

Purpose  of  Studv 

Significant  Results 

-  Watkins 
(1982) 

Navy 
(3  R&D) 

Fitted  transformed  cost  data  to 

Rayleigh  distribution  using  least 
squares  linear  regression; 

Generated  EACs  using  resulting 
regression  equations 

Coefficients  of  determination  (r2)  > 
0.815  with  all  data  included  in  the 
regression; 

EACs  highly  variable  over  the  life 
of  a  contract; 

No  comparison  made  to  other  EAC 
methods 

Abemethy 

(1984) 

Navy 
(21  R&D) 

Fitted  transformed  cost  data  to 

Rayleigh  distribution  using  both  least 
squares  and  three  group  linear 
regression; 

Determined  if  Rayleigh  parameters 
from  regressions  on  yearly  subsets  of 
data  converged  on  final  regression 
parameters  from  complete  data  sets 

Using  Kolmogorov-Smimov 
goodness-of-fit  test  with  5% 
significance  level,  Rayleigh 
distribution  found  to  be  a  good  fit 
for  14  of  the  21  contracts; 
Convergence  of  regression 
parameters  demonstrated  on  only 
one  contract 

Gallagher 
and  Lee 
(1995) 

Air  Force 
(2  R&D) 

Navy 
(1  R&D) 

Described  application  of  Multiple 
Model  Adaptive  Estimation  (MMAE) 
and  Rayleigh  distribution  to  generate 
likelihood  curves  for  final  cost  and 
completion  time  on  contracts; 
Compared  EACs  calculated  with 
Rayleigh  MMAE  technique  to  EACs 
calculated  using  CPI3,  CPI6,  CPIcum, 
and  SCIcum 

Comparisons  made  only  on 
individual  contracts; 

No  specific  measure  of  EAC 
accuracy  was  used; 

Method  producing  EAC  closest  to 
actual  CAC  varied  depending  on 
contract  and  number  of  years  of 
data  used  to  calculate  the  EACs; 

The  two  earlier  studies  by  Watkins  and  Abemethy  used  the  same  basic  form  of  the 
Rayleigh  distribution. 


C(t)  =  K(1  -e'*2 ) 


(10) 
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where  C(t)  is  the  cumulative  cost  (ACWPcum)  expended  at  time  t  ,  K  is  total  cost  at 
contract  completion,  a  is  the  Rayleigh  shape  parameter  which  governs  the  time  of  peak 
contract  expenditure,  and  t  is  the  elapsed  time  from  start  of  contract.  Both  Watkins  and 
Abemethy  used  quarterly  data  and  expressed  elapsed  time  in  quarters  (any  unit  of  time 
can  be  used).  By  taking  the  first  derivative  of  the  cumulative  Rayleigh  function,  an 
equation  expressing  the  rate  of  change  of  cumulative  contract  cost  with  respect  time  is 
obtained. 


dC(t) 

dt 


2Katea'2 


(11) 


where  dC(t)  is  the  period  cost  expenditure  (the  period  ACWP),  dt  is  the  length  of  the 
reporting  period,  K  is  total  cost  at  contract  completion,  a  is  the  Rayleigh  shape 
parameter  which  governs  the  time  of  peak  contract  expenditure,  and  t  is  the  elapsed  time 
from  start  of  contract. 


In  Watkin’s  study,  the  term  dt  was  omitted  from  his  calculations.  Because  all  of  his 
data  were  spaced  in  even  quarterly  increments  (i.e.,  dt  =  1  )  there  was  no  effect  on  the 
results  of  his  analysis.  It  is  critical,  however,  to  ensure  that  the  term  is  not  omitted  if  the 
length  of  the  reporting  period  is  anything  other  than  one  time  unit  as  was  the  case  in  this 
study  where  days  rather  than  quarters  were  used  as  the  time  unit. 

Next,  Equation  (1 1)  is  divided  by  t  to  yield 


By  taking  the  natural  logarithm 
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ln(^).ln(2Ka)-at> 


an  equation  in  the  linear  form  Y=A+BX  is  obtained,  where 


X  =  t2 


Y  =  In 


dC(t)/dt 

V  t  J 


A  =  intercept  =  ln(2Ka),  and 


B  =  slope  =  -a 


(13) 


(14) 

(15) 

(16) 
(17) 


The  slope  and  intercept  can  then  be  determined  by  regressing  Y  ,  as  calculated  in 
Equation  (15),  against  t2  ,  where  dC(f)  is  the  reporting  period  ACWP,  dt  is  the  elapsed 
time  in  the  reporting  period,  and  t  is  the  elapsed  time  from  the  start  of  the  contract. 

Both  Watkins  and  Abemethy  applied  standard  least  squares  linear  regression,  details 
of  which  can  be  found  in  any  basic  statistics  book.  However*  Abemethy  also  used  a  more 
robust  regression  technique  known  as  “three  group  linear  regression.”  An  explanation 
can  be  found  in  Understanding  Robust  and  Exploratory  Data  Analysis,  by  Hoaglin, 
Mosteller,  and  Tukey  (1983). 

Once  the  slope  and  intercept  have  been  determined  using  one  of  these  regression 
techniques,  calculating  the  EAC  is  a  simple  matter.  As  noted  earlier,  K  is  the  total  cost 
at  contract  completion  and  is,  therefore,  the  parameter  of  interest.  By  algebraically 
manipulating  Equations  (16)  and  (17),  K  can  be  expressed  in  terms  of  the  know  slope 
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(B)  and  intercept  (A)  determined  by  the  regression,  and  the  EAC  can  be  calculated.  The 
mathematical  expression  resulting  from  this  manipulation  is  as  follows: 


EAC  =  K  =  eA/(-2B) 


(18) 


Watkins  was  successful  in  using  least  squares  linear  regression  to  fit  the  non- 
cumulative  Rayleigh  distribution  to  actual  data  from  three  Navy  contracts.  In  each  of  the 
three  contracts  analyzed,  the  resulting  coefficient  of  determination  (r2)  was  at  least  0.815 
or  higher  when  all  of  the  data  were  included  in  the  regression  (Watkins,  1982).  Watkins 
also  tested  the  sensitivity  of  his  results  to  the  effect  of  inflation  on  one  of  the  contracts  by 
deflating  the  data  to  constant  1 972  dollars  and  performing  the  regression  on  the  deflated 
data.  The  regression  on  the  deflated  data  resulted  in  only  a  slightly  better  fit;  r2  increased 
from  0.815  to  0.827  (Watkins,  1982). 

Watkins  also  attempted  to  generate  EACs  using  the  resulting  regression  equations. 
However,  these  EACs  varied  significantly  from  one  quarter  to  the  next  and  even  resulted 
in  negative  values  during  the  early  stages  of  one  of  the  contracts.  Although  Watkins  was 
not  successful  in  generating  EACs  using  this  method,  the  study  did  demonstrate  the 
feasibility  of  modeling  contract  expenditures  using  the  Rayleigh  distribution. 

For  his  analysis,  Abemethy  used  both  regression  techniques  and  fit  data  from  21 
Navy  contracts  to  the  non-cumulative  Rayleigh  distribution.  Using  the  Kolmogorov- 
Smimov  goodness-of-fit  test  with  a  significance  level  of  five  percent,  Abemethy  found 
that  in  14  of  the  21  contracts  the  Rayleigh  distribution  was  a  good  fit  for  the  actual  data. 
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with  neither  regression  technique  demonstrating  clear  superiority  over  the  other 
(Abemethy,  1984). 

Abemethy  also  tested  the  ability  of  the  Rayleigh  model  to  predict  future  contract 
performance  by  comparing  the  Rayleigh  parameters  from  regressions  done  on  yearly 
subsets  of  contract  data  with  the  parameters  from  the  regressions  done  on  the  complete 
data  sets.  The  purpose  of  the  test  was  to  determine  if  the  regression  parameters  from  the 
yearly  subsets  of  contract  data  converged  on  the  final  regression  parameters  from  the 
complete  data  sets.  The  results  of  this  testing  demonstrated  convergence  in  only  one  of 
the  contracts  (Abemethy,  1984).  Unfortunately,  no  comparisons  were  made  to  the  actual 
CAC  or  to  other  EAC  methods;  and  there  is  no  way  to  determine  the  ability  of  the 
techniques  to  calculate  EACs  based  on  the  results  of  this  study. 

The  most  recent  study  involving  the  Rayleigh  distribution,  done  by  Gallagher  and 
Lee  (1995),  describes  a  methodology  that  uses  Multiple  Model  Adaptive  Estimation 
(MMAE)  to  determine  a  range  values  for  the  Rayleigh  shape  parameters  and  the 
probability  that  any  one  set  of  parameters  is  correct  conditional  on  the  actual  cost  data.  A 
description  of  the  MMAE  technique  can  be  found  in  Stochastic  Models,  Estimation,  and 
Control:  Volume  2  (Maybeck,  1982).  This  range  of  parameters  and  their  associated 
probabilities  are  then  used  to  generate  a  point  estimate  for  the  EAC,  as  well  as  probability 
distributions  for  the  most  likely  range  of  final  contract  cost  and  completion  time. 

The  authors  tested  the  method  using  data  from  three  DoD  R&D  contracts  (two  Air 
Force  and  one  Navy)  by  plotting  cumulative  cost  probability  curves  generated  with  yearly 
increments  of  data  and  comparing  these  plots  to  the  actual  contract  cost  at  completion. 
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No  specific  measure  of  accuracy  was  used;  however,  an  examination  of  the  plots  indicates 
that  the  estimated  range  was  reasonably  accurate  after  three  to  four  years  of  data  were 
included  (Gallagher  and  Lee,  1995). 

The  authors  also  calculated  EACs  using  four  index-based  methods  (CPI3,  CPI6, 
CPIcum,  and  SCIcum)  which  they  compared  to  the  point  estimate  EACs  calculated  with 
their  method.  For  each  method,  EACs  were  calculated  for  yearly  increments  of  data 
starting  with  either  the  second  or  third  year  of  contract  completion,  and  continuing 
through  the  final  year  of  each  contract.  Unfortunately,  no  specific  measure  of  accuracy 
was  used  to  compare  the  methods.  A  comparison  of  the  EACs  calculated  using  each  of 
the  methods  indicates  that  the  method  producing  the  EAC  closest  to  the  actual  contract 
CAC  varied  depending  on  the  contract  and  the  number  of  years  of  data  used  (Gallagher 
and  Lee,  1995).  Unfortunately,  due  to  the  limited  number  of  contracts  (three),  and  the 
fact  that  no  specific  measure  of  accuracy  was  used  for  the  comparisons,  it  is  difficult  to 
draw  any  meaningful  conclusions  from  the  results  of  this  study. 

A  detailed  description  of  this  method  is  beyond  the  scope  of  this  study  and  can  be 
found  in  Gallagher  and  Lee’s  paper  (1995).  A  copy  of  the  software  necessary  to  calculate 
EACs  using  this  method  was  obtained  from  the  authors  and  is  one  of  the  EAC  methods 
evaluated  in  this  study.  A  copy  of  the  Microsoft®  Excel  Visual  Basic®  code  used  to 
implement  the  method  is  included  as  Appendix  E  of  this  paper. 

The  ability  of  the  Rayleigh  curve  to  accurately  model  actual  contract  cost 
expenditures  was  demonstrated  in  each  of  the  three  studies  reviewed;  however,  only  the 
Gallagher  and  Lee  study  attempted  a  comparison  to  other  EAC  methods,  with 
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inconclusive  results.  Therefore,  the  three  nonlinear  methods  using  the  Rayleigh 
distribution,  1)  least  squares  linear  regression  (Abemethy,  1984;  Watkins,  1982),  2)  three 
group  linear  regression  (Abemethy  1984),  and  3)  MMAE  (Gallagher  and  Lee,  1995)  were 
included  in  this  study. 

Beta  Curve  EAC  Method.  The  second  nonlinear  model  was  the  Beta  distribution. 

An  EAC  method  using  the  Beta  distribution  was  presented  by  Whitlock  of  the  NASA 
Johnson  Space  Center  in  the  Fall  1982  issue  of  the  Journal  of  Parametrics.  This  method 
makes  use  of  a  modified  Beta  function  with  the  following  cumulative  cost  curve: 

C(t)  =  A(  1 0t2-20t3+ 1 0t4)+B(  1 0t3-20t4+ 1 0t5)+(5t4-4t5)  (19) 

where  C(t)  is  the  fraction  of  total  contract  cost  consumed  at  time  t  ,  t  is  the  fraction  of 
time  elapsed  in  the  program,  which  is  equal  to  the  elapsed  time  as  of  the  report  date 
divided  by  the  expected  duration  of  the  contract,  and  A  and  B  are  the  weighting  factors 
of  the  cumulative  Beta  function,  subject  to  the  constraints  A  >  0  ,  B  >  0  ,  A+B  <  1  . 

To  determine  the  parameters  (A  and  B)  that  achieve  the  best  fit  to  the  actual  cost 
data,  Whitlock  proposed  minimizing  a  weighted  sum  of  the  least  squares  (WSLS) 
expression  with  fractional  time  cubed  used  as  the  weighting  factor.  Fractional  time  cubed 
was  chosen  as  the  weighting  factor  because  “the  spending  pattern  becomes  better  defined 
the  later  into  a  program  you  are”  (Whitlock,  1982:17). 

The  WSLS  is  expressed  as  follows: 


WSLS  =  E[C(tj)  -  YKjjV 


(20) 
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where  C(t)  is  the  fractional  cumulative  cost  curve  as  defined  in  Equation  (19),  Y  is  a 
scaling  factor  which  expresses  cumulative  program  cost  (Kj)  as  a  fractional  cost,  Kj  is 
the  cumulative  program  cost,  or  ACWPcum»  for  the  ith  reporting  period,  and  tj  is  the 
fractional  time  elapsed  as  of  the  ith  reporting  period. 

To  solve  for  the  three  unknowns  (A,  B,  and  Y)  Whitlock  proposed  four 
computational  techniques,  three  of  which  were  not  practical  for  the  purposes  of  this  study. 
The  first  of  these  three  techniques  involved  selecting  values  for  A  and  B  based  on 
knowledge  of  the  program,  or  of  similar  programs,  and  then  solving  for  Y.  This 
technique  was  determined  to  be  impractical  because  it  requires  a  level  of  familiarity  with 
the  program  beyond  the  knowledge  of  this  researcher  and  the  information  available  in  the 
DAES  database. 

The  second  technique  involves  setting  up  a  system  of  linear  equations  in  three 
unknowns  using  partial  derivatives  and  then  solving  the  system  of  equations.  This 
technique  was  deemed  impractical  because  it  could  not  be  easily  automated.  Given  the 
large  volume  of  data  to  be  analyzed  and  the  time  constraints  in  this  study,  automation  was 
considered  to  be  essential. 

The  third  technique  involved  the  use  of  quadratic  programming  to  minimize  the 
WSLS.  Although  Microsoft©  Excel  has  the  capability  to  solve  quadratic  programming 
problems,  the  process  is  not  easily  automated.  Therefore,  this  technique  was  also  deemed 
impractical. 

The  fourth  and  most  practical  technique  for  the  purposes  of  this  study  was  an 
exhaustive  search  technique  that  involved  writing  a  computer  algorithm  to  test  various 
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values  of  the  weighting  factors  A  and  B,  in  discrete  increments,  until  the  smallest  WSLS 
value  is  found.  This  computer  algorithm  was  easily  implemented  in  Microsoft®  Excgj 
Visual  Basic®  and  is  fully  discussed  in  Chapter  III. 

With  the  exhaustive  search  technique,  the  scaling  factor  Y  is  calculated  using 


EC(t,)K,«f  (21) 

W  £,,K3  ’ 

which  is  obtained  by  setting  the  first  derivative  of  Equation  (20)  equal  to  zero  and  solving 
forY. 

Once  the  values  of  the  parameters  A,  B,  and  Y  are  determined,  the  final  step  is 
calculating  the  EAC.  The  following  relationship  exists  for  any  data  point. 

YKj  =  C(tj)  (22) 

Using  this  relationship,  an  estimate  of  cumulative  program  cost  can  be  developed  for  any 
future  point  in  time.  At  the  point  in  time  corresponding  to  the  end  of  the  program, 

K  =  Cftenj)  (23) 


However,  Kend=EAC  ,  and  C(t€nd)-C(l)-1  .  Therefore, 

EAC  =  — 

Y 


(24) 
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Unfortunately,  Whitlock  made  no  attempt  to  verify  the  accuracy  of  this  method  or  to 
compare  its  accuracy  with  other  EAC  methods. 

Conclusions 

This  review  of  previous  research  demonstrates  several  key  points.  First,  the  index- 
based  EAC  formulas  are  widely  used,  have  been  extensively  tested,  and  provide  an 
excellent  basis  for  comparing  newer,  and  possibly  better  EAC  methods  that  are  being 
developed.  Because  the  linear,  index-based  methods  are  easy  to  use  they  will  likely 
continue  to  be  the  primary  method  of  calculating  EACs  well  into  the  future.  In  addition, 
all  of  the  EAC  methods  included  in  the  DoD’s  Performance  Analyzer  software  package 
are  linear  methods  (Cost  Management  Systems,  Inc.,  1994)  demonstrating  the  popularity 
of  the  index-based  methods. 

Second,  the  accuracy  of  a  given  EAC  method  is  dependent  on  such  factors  as 
contract  completion  stage,  system  type,  and  program  phase.  Including  a  sensitivity 
analysis  to  test  for  the  effects  of  such  factors  on  the  overall  results  of  a  comparative  EAC 
study  is  critical. 

Finally,  the  performance  of  nonlinear  EAC  methods  has  rarely  been  compared  to  the 
performance  of  the  index -based  methods.  In  addition,  the  relative  accuracy  of  the 
nonlinear  regression-based  EAC  methods  has  not  been  established.  For  these  reasons,  a 
comparative  study  of  nonlinear  regression-based  and  index-based  EAC  methods  is  clearly 
needed.  As  more  sophisticated  EAC  methods  are  developed,  the  only  way  they  will  gain 
acceptance  is  if  they  can  be  proven  superior  by  comparison  with  accepted  methods. 
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Whether  these  new  methods  are  proven  superior  or  not,  this  research  still  provides  insight 
into  the  relative  accuracy  of  the  various  EAC  methods. 
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III.  Methodology 


Introduction 

This  chapter  describes  the  procedures  used  to  compare  the  performance  of  15  EAC 
methods,  including  1 1  index -based  methods,  and  four  nonlinear  regression-based 
methods.  The  four  nonlinear  regression-based  methods  make  use  of  the  Rayleigh 
distribution  and  modified  Beta  distribution  discussed  in  the  previous  chapter.  The 
objective  of  this  comparison,  as  stated  earlier,  is  to  determine  if  the  complex  nonlinear 

regression-based  EAC  methods  perform  better  than  the  simpler  index-based  EAC 
methods. 

The  chapter  begins  with  a  description  of  the  database  used  to  compare  the  EAC 
methods,  and  the  criteria  that  were  used  to  screen  the  data.  Next,  the  actual  EAC 
methods  that  were  tested  are  discussed,  along  with  specific  computational  issues 
concerning  each  of  the  methods.  A  discussion  of  what  constitutes  “good”  performance 
for  an  EAC  method,  and  a  presentation  of  the  specific  performance  measures  used  to 
evaluate  the  EAC  methods  follows.  Next,  the  specific  procedures  used  to  compare  the 
methods,  and  to  test  for  sensitivity  of  the  results  to  stage  of  contract  completion,  system 
type,  program  phase,  contract  type,  DoD  service  component,  and  the  effects  of  inflation 

are  described.  Finally,  the  chapter  concludes  with  a  justification  of  the  approach  taken  in 
this  analysis. 
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The  DAES  Datahagp 

The  DAES  database  contains  cost  performance  data  on  541  completed  or  near 
completed  defense  contracts  covering  the  period  from  June  1970  to  October  1992.  This 
database  includes  both  development  and  production  contracts  from  all  three  services  and 
a  variety  of  different  system  types.  The  cost  performance  data  for  each  contract  are 
presented  as  a  series  of  periodic  (generally  quarterly)  reports.  The  data  extracted  from  the 
database  included  several  descriptive  fields  as  well  as  the  actual  cost  report  data.  The 
descriptive  fields  included  the  project  number  (PNO),  the  contract  number  (CNO),  a 
description  of  the  contract  (PD),  the  branch  of  service  administering  the  contract  (SER), 
the  contract  type  (CON),  the  program  phase  (PHA),  and  the  date  that  work  started  on  the 
contract  (WSDATE).  The  actual  cost  report  data  included  the  submittal  date  for  the  cost 
report  (SDATE),  the  Budgeted  Cost  of  Work  Scheduled  (BCWS),  the  Budgeted  Cost  of 
Work  Performed  (BCWP),  the  Actual  Cost  of  Work  Performed  (ACWP),  the  Budget  at 
Completion  (BAC),  the  Contract  Budget  Base  (CBB),  the  Total  Allocated  Budget  (TAB), 
and  the  percentage  complete  relative  to  the  final  cost  of  the  contract  (PCF). 

Data  Screening  Criteria 

In  most  cases  the  contracts  contained  in  this  database  did  not  include  a  complete  set 
of  cost  data  through  the  100  percent  completion  point,  and  in  some  cases  the  data  stops 
earlier  than  the  90  percent  completion  point.  Because  the  contract’s  CAC  was  required  as 
a  comparison  point  to  evaluate  the  performance  of  each  EAC  method,  an  objective 
measure  of  the  CAC  was  needed.  For  the  purposes  of  this  study,  a  contract’s  CAC  was 
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defined  as  the  AC  WPcum  as  of  the  final  set  of  data  for  a  given  contract,  and  any  contract 
not  having  data  for  the  90  percent  completion  point  or  greater  was  eliminated  from  the 
analysis.  Ideally,  only  contracts  with  data  through  the  100  percent  completion  point 
would  have  been  used;  however,  this  would  have  severely  limited  the  number  of  contracts 
analyzed  in  the  study  (only  10  contracts  after  satisfying  the  remaining  screening  criteria). 

A  second  data  screening  criterion  involved  the  elimination  of  contracts  without  data 
for  the  period  earlier  than  the  10  percent  completion  point.  This  was  necessary  because 
of  the  need  to  evaluate  the  performance  of  the  EAC  methods  over  the  entire  life  of  a 
contract.  As  mentioned  previously,  a  major  purpose  of  the  EAC  is  to  provide  senior 
managers  with  feedback  to  allow  early  identification  and  correction  of  problems.  For  this 

reason,  the  evaluation  of  the  EAC  methods  during  the  early  stages  of  contract  completion 
was  critical. 

A  third  screening  criterion  involved  the  need  for  a  definitive  date  that  work  started  on 
the  contract.  This  was  necessary  because  all  of  the  regression-based  methods  required  a 
known  start  date  to  calculate  the  EAC.  For  this  reason,  any  contracts  that  did  not  include 
a  start  date,  any  contracts  that  listed  multiple  start  dates,  and  any  contracts  that  listed  start 
dates  occurring  after  the  initial  cost  report  submittal  date  were  also  eliminated  from  the 
analysis.  Another  note  regarding  the  start  dates  is  in  order.  The  start  dates  listed  in  the 
database  included  only  the  month  and  year.  This  required  an  assumption  to  be  made 
regarding  the  actual  date  that  work  started  on  the  contract.  Without  knowledge  of 
whether  a  given  contract  started  early  or  late  in  the  month  the  best  way  to  minimize  the 
potential  for  error  was  to  assume  that  all  of  the  contracts  began  on  the  1 5th  of  the  month. 
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The  fourth  screening  criterion  involved  contracts  with  that  went  over  target  baseline 
(OTB),  that  is,  contracts  where  the  contractor  is  “authorized  to  implement  and  report  to  a 
baseline  or  Total  Allocated  Budget  (TAB)  that  exceeds  the  cost  of  authorized  work  or 
Contract  Budget  Base  (CBB)”  (Department  of  the  Air  Force,  1993:1  l-B-2).  Because 
OTB  contracts  occur  only  in  exceptional  cases  (Department  of  the  Air  Force,  1993),  and 
because  calculating  an  EAC  for  a  contract  that  has  gone  OTB  is  difficult  without  a  full 
understanding  of  the  cause,  any  contracts  where  the  CBB  did  not  equal  the  TAB  were 
eliminated  from  the  analysis. 

The  final  data  screening  criterion  involves  some  peculiarities  that  occurred  with  14 
contracts  that  complicated  the  calculation  of  the  index-based  EACs.  Specifically,  these 
peculiarities  involved  contracts  where  either  the  BCWS,  BCWP,  or  ACWP  decreased 
from  one  reporting  period  to  the  next.  In  these  cases,  it  was  not  possible  to  calculate  the 
index-based  EACs  that  required  the  use  of  the  cost  parameter  that  decreased.  In  order  to 
simplify  the  analysis,  and  because  the  number  of  contracts  where  this  situation  occurred 
was  limited,  these  contracts  were  also  eliminated  from  the  analysis. 

After  the  data  were  screened  based  on  the  criteria  discussed  above,  a  total  of  88 
contracts  remained.  These  contracts  along  with  their  descriptive  data  are  listed  in 
Appendix  B.  With  the  data  screening  complete,  the  next  step  was  to  calculate  the  EACs. 

EAC  Methods  Tested 

As  discussed  in  Chapter  II,  the  two  basic  categories  of  EAC  methods  relevant  to  this 
study  were,  the  index-based  methods,  and  the  nonlinear  regression-based  methods.  This 
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section  describes  all  of  the  methods  that  were  tested  in  this  study,  as  well  as  specific 
computational  issues  associated  with  each  method. 

Index-Based  EAC  Methods.  Because  of  the  ease  of  calculating  the  index-based 
EACs,  and  the  popularity  of  these  methods,  1 1  different  indices  were  analyzed.  Nine  of 
these  indices  included  the  CPI,  SPI,  and  SCI  in  both  their  cumulative  form  (CPIcum, 
SPIcum,  and  SCIcum),  as  defined  Equations  (2),  (5),  and  (8)  respectively,  and  using  the 
three,  and  six  month  moving  average  indices  (CPI3,  CPI6,  SPI3,  SPI6,  SCI3,  and  SCI6),  as 
defined  in  Equations  (3)  and  (6),  and  (8).  The  tenth  index  was  the  modified  form  of  the 
SCI  where  CPI6  is  multiplied  by  SPIcum.  This  form  of  the  SCI  is  similar  to  the  form  found 
to  be  the  most  accurate  overall  EAC  by  Bright  and  Howard  in  their  1981  study  (they 

actually  used  CPI6  rather  than  CP16),  and  is  also  one  of  the  methods  included  in 
Performance  Analyzer  (Cost  Management  Systems,  Inc.,  1994).  The  final  index  tested 
was  the  Composite  Index,  defined  in  Equation  (9),  with  SPIcum  multiplied  by  a  weighting 
factor  (W|)  of  0.2  and  added  to  CPIcum  multiplied  by  a  weighting  factor  (W2)  of  0.8 
(0.2SPIcum+0.8CPIcum).  This  is  the  same  index  that  was  found  to  perform  well  on  some 
system  types,  at  some  stages  of  contract  completion  in  the  1989  study  by  Reidel  and 
Chance.  This  method  is  also  highly  recommended  in  Air  Force  Materiel  Command’s 
Financial  Management  Guide  to  Analysis  of  Contractor  Cost  Data  (1994). 

EACs  were  calculated  for  every  cost  reporting  period  on  all  contracts  using  each  of 
the  1 1  index-based  methods,  except  in  the  following  cases.  First,  the  three  and  six  month 
indices  could  not  be  calculated  for  reporting  periods  that  occurred  less  than  three  or  six 
months  into  the  contract.  And  second,  EACs  could  not  be  calculated  in  cases  where  a 
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division  by  zero  occurred.  This  happened  in  the  following  situations:  for  EAC  methods 
involving  the  moving  average  CPIs  when  the  period  ACWP  was  equal  to  zero,  or 
methods  involving  CPIcum  when  ACWPcum  was  equal  to  zero;  for  EAC  methods 
involving  the  moving  average  SPIs  when  the  period  BCWS  was  equal  to  zero,  or  methods 
involving  SPIcum  when  BCWScum  was  equal  to  zero;  and,  for  all  EAC  methods  when 
either  the  period  BCWP  or  BCWPcum  was  equal  to  zero.  Calculation  of  all  index-based 
EACs  was  done  with  a  user  defined  function  written  in  Microsoft®  Excel  Visual  Basic® 
(Microsoft®  Corporation,  1993).  A  copy  of  the  user  defined  function  is  included  as 
Appendix  C. 

There  were  two  issues  involved  with  calculating  the  index-based  EACs  using  the 
data  from  the  DAES  database.  First,  because  the  BCWS,  BCWP,  and  ACWP  data 
included  in  the  database  are  cumulative,  calculating  the  three  and  six  month  indices 
required  special  treatment  (the  cumulative  indices  could  be  calculated  directly  from  the 
cumulative  cost  number  included  in  the  database  using  the  equations  previously 
mentioned).  In  cases  where  the  reporting  periods  were  spaced  in  even  three  and  six 
month  intervals  the  calculations  involved  only  the  following  equations: 


_  BCWPcum  (tnow )  -  BCWPcum  (tnow  -  n) 
ACWPcum  (tnow )  -  ACWPcum  (tnow  -  n) 


(25) 


BCWPcum  (tnow )  -  BCWPcum  (tnow  -  n) 
BCWScuni  (tnow )  -  BCWScum  (tnow  -  n) 


(26) 
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where  W  is  the  current  period  for  which  the  index  is  being  calculated  and  n  is  the 
number  of  months  over  which  the  index  is  being  calculated  (either  three  or  six  months). 
Where  actual  data  existed,  these  indices  could  be  calculated  directly  using  these  two 
equations.  However,  ihe  second  issue  occurred  in  cases  where  a  report  was  missing  for 
the  period  either  three  or  six  months  prior  to  the  current  period,  either  because  is  was  not 
submitted,  or  because  of  odd  reporting  cycles.  In  these  cases  the  missing  data  had  to  be 
interpolated.  To  minimize  the  number  of  data  points  that  required  interpolation,  a 
criterion  of  ±15  days  from  the  actual  three  or  six  month  prior  point  was  established.  That 
is,  if  there  was  an  actual  report  with  a  report  date  within  ±15  days  of  the  date  three  or  six 
months  prior  to  the  report  date  for  which  the  EAC  was  being  calculated,  the  actual  data 
from  that  report  were  used.  When  interpolation  was  required,  simple  linear  interpolation 
was  used.  Because  there  is  no  way  to  know  the  actual  nature  of  the  cost  expenditures 
over  the  missing  report  period,  any  interpolation  technique  will  provide  at  best  an 
approximation  of  the  actual  values.  Therefore,  to  ease  computations  the  simplest  method 
available,  linear  interpolation,  was  chosen.  Interpolation  of  the  missing  data  points  was 
done  using  the  following  equation: 

c  =(C+-C-Xtx-Q  +  C  (27) 

x  t+-t_ 

where  Cx  is  the  unknown  cost  value  (BCWS,  BCWP,  or  ACWP)  for  the  missing  report, 

with  a  corresponding  time,  tx  ,  C+ is  the  actual  cost  value  for  the  period  just  after  the 
missing  report,  with  a  corresponding  time,  t+  ,  and  C.  is  the  actual  cost  value  for  the 
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period  just  prior  to  the  missing  report,  with  a  corresponding  time,  t.  .  An  example 
should  help  to  illustrate  this: 

t.  =  25  April  77  BCWP.  =  $53  million 

tx  =  25  May  77  BCWPX  =  ? 

t+ =  25  August  77  BCWP+ =  $122  million 

In  this  case  the  missing  data  for  three  months  prior  to  25  August  77,  which  is  25  May  77, 
needs  to  be  interpolated.  The  difference  between  tx  and  t.  is  30  days,  and  the  difference 
between  t+  and  t.  is  122  days.  Using  Equation  (27): 

BCWP  =  ($1221$53X3°)  +  $53  =  $69  97 
x  122 

an  interpolated  BCWP  value  of  $70  million  (rounded  off)  is  calculated.  Of  course  to 
calculate  the  performance  indices,  all  three  cost  values  (BCWS,  BCWP,  and  ACWP) 
must  be  calculated.  The  interpolated  cost  values  were  then  used  in  Equations  (25)  and 
(26)  in  place  of  the  missing  data.  Calculation  of  the  interpolated  data  points,  and  the  non- 
cumulative  three  and  six  month  values  for  BCWS,  BCWP,  and  ACWP  was  done  as  part 
of  the  same  user  defined  function  (Appendix  C)  used  to  calculate  the  index -based  EACs. 

Nonlinear  EAC  Methods.  Four  nonlinear  EAC  methods  were  evaluated  in  this 
study.  They  include  the  least  squares  linear  regression  method  using  the  Rayleigh 
distribution  (Abemethy,  1984;  Watkins,  1982),  the  three  group  linear  regression  method 
using  the  Rayleigh  distribution  (Abemethy,  1984),  the  MMAE  method  using  the  Rayleigh 
distribution  (Gallagher  and  Lee,  1994),  and  the  exhaustive  search  method  using  the 
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modified  Beta  distribution  (Whitlock,  1982).  The  details  of  calculating  EACs  with  each 
of  these  methods  will  be  discussed  in  the  following  paragraphs. 

Because  the  two  regression  methods  involving  the  Rayleigh  distribution  required 
non-cumulative  data,  the  first  step  was  to  calculate  the  period  ACWP  values  for  each 
reporting  period.  This  was  done  by  subtracting  the  previous  reporting  period’s  ACWPcum 
from  the  current  period’s  ACWPcum.  In  the  case  of  the  first  reporting  period,  ACWPcum  is 
also  the  period  ACWP;  therefore,  no  subtraction  was  required. 

After  calculating  the  period  ACWPs,  the  next  step  was  to  transform  the  data  so  the 
linear  regressions  could  be  done.  This  was  done  using  Equation  (15),  and  provided  the 
dependent  variable  (Y)  for  the  regression.  The  independent  variable  (X)  in  the  regression 
is  time  squared  (where  time  is  measured  from  the  start  of  the  contract).  To  calculate  the 
independent  variables  for  regression,  the  number  of  days  from  the  start  of  the  contract 
was  calculated  and  then  squared.  All  of  these  calculations  were  done  using  basic 
mathematical  manipulations  in  Microsoft®  Excel  (Microsoft®  Corporation,  1993). 

Once  the  regression  variables  had  been  calculated,  both  regression  techniques  were 
applied  to  calculate  the  intercept  (A)  and  slope  (B)  for  each  reporting  period.  The  least 
squares  linear  regression  was  done  using  the  built-in  SLOPE,  and  INTERCEPT 
commands  in  Microsoft®  Excel  (Microsoft®  Corporation,  1993).  The  three  group  linear 
regression  was  done  with  a  user  defined  function  written  in  Visual  Basic®.  A  copy  of 
this  user  defined  function  is  included  as  Appendix  D. 

After  the  slope  and  intercept  were  determined  using  each  of  the  regression  methods, 
the  EACs  were  calculated  using  Equation  (18).  The  least  squares  linear  regression 
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function  in  Microsoft®  Excel  requires 


at  least  two  data  points;  therefore,  EACs  were 


calculated  for  this  method  beginning  with  the  second  reporting  period  for  each  contra 
Because  the  three  group  linear  regression  technique  requires  a  minimum  of  three  data 
points,  EACs  were  calculated  for  this  method  beginning  with  the  third  reporting  period 
for  each  contract.  Also,  EACs  for  the  three  group  regression  method  could  not  be 
calculated  when  the  period  ACWP  was  equal  to  zero.  Other  than  the  exceptions  just 
noted,  EACs  were  calculated  for  every  cost  reporting  period  on  all  contracts  using  both 

methods. 

Calculating  EACs  with  the  Rayleigh  MMAE  method  was  done  completely  within  the 
user  defined  function  obtained  from  Gallagher.  No  special  manipulations  of  the  data 
were  required.  The  only  modification  required  to  the  user  defined  function  was  the 
substitution  of  the  inflation  indices  used  in  this  study  for  the  inflation  indices  Gallagher 
used.  A  more  detailed  discussion  of  inflation  indices  is  included  in  the  section  discussing 
the  test  for  sensitivity  of  the  results  to  the  effects  of  inflation.  A  copy  of  the  MMAE  user 
defined  function  is  included  as  Appendix  E.  EACs  were  calculated  using  this  method  for 
every  cost  reporting  period  beginning  with  the  second  cost  report  on  all  contracts.  The 
method  required  a  minimum  of  two  data  points  so  an  EAC  could  not  be  calculated  with 

only  the  first  cost  report. 

To  calculate  EACs  with  the  modified  Beta  distribution  proposed  by  Whitlock  (1982), 
a  user  defined  function  was  developed  using  YisuaLBasic®  (included  as  Appendix  F). 
The  inputs  to  this  function  were  fractional  time  and  ACWP,„.  Calculating  fractional 
time  required  a  known  end  date  for  each  contract.  Because  this  information  was  not 
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available  in  the  DAES  database,  the  submittal  date  for  the  final  cost  report  was  used. 
Calculating  fractional  time  was  done  by  dividing  the  number  of  days  elapsed  from  the 
start  of  the  contract  as  of  the  report  submittal  date  by  the  total  number  of  days  in  the 
contract.  The  user  defined  function  was  then  used  to  determine  the  values  of  the 
parameters  A  and  B  [Equation  (19)]  that  minimized  the  WSLS  [Equation  (20)].  This  was 
done  by  testing  various  values  of  A  and  B  (subject  to  the  constraints  A  >  0  ,  B>0  ,  and 
A+B<1  )  first  in  increments  of  0.1,  and  then  in  increments  of  0.01  around  the  initial  set 
of  parameters.  Once  the  values  of  A  and  B  producing  the  minimum  WSLS  were 
determined,  the  value  of  Y  was  calculated  using  Equation  (21),  followed  by  the  EAC 
using  Equation  (24).  As  with  the  other  methods,  an  EAC  was  calculated  for  every  cost 
reporting  period  on  all  contracts.  The  only  case  where  the  Beta  EAC  could  not  be 
calculated  was  in  periods  where  ACWPcum  was  equal  to  zero. 

EAC  Evaluation  Criteria 

After  the  EACs  were  calculated,  the  next  step  was  to  compare  the  methods  to 
determine  which  ones  produced  the  best  estimates.  Covach,  et  cil..  provide  three  criteria 
for  evaluating  the  performance  of  EAC  methods. 

Accuracy  -  A  method’s  estimate  of  costs  at  completion  (EAC)  should 
generally  be  equal  or  close  to  the  contractor  actual  cost  a  completion  (CAC). 

Timeliness  -  A  method  should  be  capable  of  producing  a  reliable  EAC  as 
early  as  possible  in  the  life  of  the  contract. 

Stability  -  A  method  should  not  produce  EACs  which,  on  a  month  to  month 
basis,  vary  widely.  (Covach,  et  al.,  1981 :21) 
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These  three  criteria  provide  a  good  basis  for  evaluating  the  performance  of  EAC  methods. 
The  specific  measures  that  were  used  to  compare  the  performance  of  the  EAC  methods 
will  be  discussed  in  the  next  section. 

Measures  of  EAC  Performance 

Accuracy.  Several  different  measures  are  available  to  test  the  accuracy  of  an 
estimating  method.  Two  of  these,  suggested  by  Nahmias,  are  the  Mean  Absolute 
Deviation  (MAD)  and  the  Mean  Squared  Error  (MSE)  (1993).  The  problem  with  using 
either  of  these  measures  is  that  they  are  dependent  on  the  magnitude  of  the  error,  and 
therefore  are  not  useful  for  comparing  results  across  series  of  data  with  differing 
magnitudes.  For  example,  an  error  of  $  1 0  million  would  be  highly  significant  on  a 
contract  with  a  final  cost  of  $50  million;  however,  this  error  would  be  minimal  on  a 
contract  with  a  final  cost  of  $1  billion.  Because  the  final  costs  of  the  contracts  in  the 
DAES  database  are  so  variable,  a  measure  that  was  not  dependent  on  the  magnitude  of 
the  error  was  required.  In  this  situation,  Nahmias  suggests  the  use  of  the  Mean  Absolute 
Percentage  Error  (1993).  Makridakis  and  Wheelwright  also  suggest  the  use  of  the  MAPE 
rather  than  the  Mean  Percentage  Error  (MPE)  because  the  tendency  of  the  MPE  is  to 
underestimate  the  true  magnitude  of  the  errors  due  to  the  cancellation  effects  of  positive 
and  negative  error  values  (1978).  The  use  of  the  MAPE  to  compare  the  accuracy  of 
estimating  methods  is  also  supported  by  Armstrong  and  Collopy.  The  major  drawback  to 
the  MAPE  suggested  in  their  study  is  that  it  is  only  relevant  for  data  with  a  meaningful 
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zero  (Armstrong  and  Collopy,  1992).  Because  a  zero  value  is  relevant  for  the  data  in  this 
study,  this  drawback  is  not  a  factor. 

To  calculate  the  MAPE  for  a  series  of  EACs,  the  first  step  is  to  calculate  the 
Absolute  Percentage  Error  (APE)  for  each  EAC.  This  is  done  using  the  following 
formula: 


APE  = 


EAC  -  CAC 
CAC 


(28) 


where  EAC  is  the  Estimate  at  Completion  for  a  given  cost  reporting  period,  and  CAC 
is  the  Cost  at  Completion  of  the  contract  for  which  the  EAC  was  calculated.  The  MAPE 
is  then  calculated  for  a  given  data  series  using  the  following  formula: 


I  ape, 

MAPE  =  — -  p< 

n 

where  APEj  is  the  Absolute  Percentage  Error  of  the  ith  EAC  in  the  series,  and  n  is  the 
total  number  of  estimates  included  in  the  series  for  which  the  MAPE  is  being  calculated. 

Timeliness.  No  specific  measure  of  the  timeliness  was  developed.  However,  the 
timeliness  of  the  EAC  methods  will  be  examined  by  separately  analyzing  the  data  during 
four  distinct  stages  of  contract  completion.  These  four  stages  include:  zero  to  25  percent 
complete;  greater  than  25  percent  to  50  percent  complete;  greater  than  50  percent  to  75 
percent  complete;  and,  greater  than  75  percent  to  100  percent  complete. 
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Stability.  None  of  the  comparative  EAC  studies  or  material  on  forecasting  methods 
reviewed  suggested  any  measures  to  test  the  stability  of  an  estimating  method.  The 
common  measure  of  the  variability  of  a  data  series  used  by  statisticians  is  the  standard 
deviation  (Devore,  1991);  therefore,  the  measure  chosen  for  this  study  was  the  Standard 
Deviation  of  the  Absolute  Percentage  Error  (SDAPE).  Calculating  the  SDAPE  for  a 
series  of  data  is  done  using  the  following  formula: 


SDAPE  = 


1 


£(APE,  -  MAPE)2 


i=l 


n-1 


(30) 


where  APE,  is  the  Absolute  Percentage  Error  of  the  ith  EAC  in  the  series,  MAPE  is  the 
Mean  Absolute  Percentage  Error  for  the  overall  data  series,  and  n  is  the  total  number  of 
estimates  included  in  the  series  for  which  the  MAPE  and  SDAPE  are  being  calculated. 

Combined  Measure  of  Performance.  Because  each  method  was  being  evaluated 
based  on  its  performance  relative  to  two  different  measures  (MAPE  and  SDAPE),  a 
combined  measure  was  needed  to  directly  compare  the  overall  performance  of  the 
methods.  This  was  done  by  ranking  the  methods  according  to  increasing  values  of  the 
MAPE  and  then  according  to  increasing  values  of  SDAPE.  Once  the  rankings  were  done, 
each  method  was  assigned  two  separate  scores,  one  according  to  its  position  in  the 
rankings  for  the  MAPE,  and  one  according  to  its  position  in  the  rankings  for  the  SDAPE. 
If  a  method  had  the  lowest  MAPE  or  SDAPE  it  received  a  score  of  one.  The  method  with 
the  next  highest  value  of  MAPE  or  SDAPE  received  a  score  of  two,  and  so  on,  until  all  of 
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the  methods  received  a  score  for  MAPE  and  a  score  for  SDAPE.  In  cases  where  two  or 
more  methods  had  equal  values  for  the  MAPE  or  SDAPE,  all  of  the  tied  methods 
received  the  same  score,  and  the  method  with  the  next  highest  MAPE  or  SDAPE  value 
was  assigned  the  next  highest  score.  For  example,  if  two  methods  were  tied  with  the 

lowest  MAPE,  both  would  receive  a  score  of  one.  The  method  with  the  next  highest 
MAPE  then  received  a  score  of  two. 

After  the  rankings  were  done  and  the  scores  assigned,  the  combined  measure  of 
performance  was  calculated  by  adding  the  two  individual  scores  (one  for  MAPE  and  one 
for  SDAPE)  together.  The  lowest  possible  score  was  two,  which  was  achieved  if  a 
method  had  both  the  lowest  MAPE  and  the  lowest  SDAPE.  The  highest  possible  score 
was  30  which  would  only  be  achieved  if  there  were  no  ties  and  the  method  had  the 
highest  MAPE  and  highest  SDAPE. 

Test  Procedures 

When  comparisons  are  made  between  data  sets  the  typical  approach  is  to  use  one  of 
several  statistical  techniques  such  as  the  t-test  or  ANOVA.  The  drawback  to  these 
techniques  is  that  they  require  independent  data  sets  for  the  results  to  be  valid  (Devore, 
1991).  In  this  study  the  condition  of  independence  is  not  met,  because  the  same  set  of 
data  is  used  for  calculating  the  EACs  with  all  of  the  methods.  Because  this  condition  of 
independence  was  not  satisfied,  a  direct  comparison  of  the  combined  measure  of 

performance  discussed  in  the  previous  section  was  used  to  determine  which  methods 
produced  the  best  EACs. 
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Evaluation  of  Performance  With  All  Contracts  Included  To  compare  the 
performance  of  the  EAC  methods  overall,  the  MAPE  and  SDAPE  for  each  method  was 
calculated  with  all  data  points  included.  The  combined  score,  as  described  above,  was 

then  calculated  for  each  method,  and  the  method  with  the  lowest  score  was  classed  as  the 
best  performing  method. 

Sensitivity  of  Results  to  Stage  of  Contract  Completion  The  sensitivity  of  the  results 
to  stage  of  contract  completion  was  tested  at  four  stages  (quartiles):  zero  to  25  percent 
complete  (first  quartile);  greater  than  25  percent  to  50  percent  complete  (second  quartile); 
greater  than  50  percent  to  75  percent  complete  (third  quartile);  and,  greater  than  75 
percent  to  100  percent  complete  (fourth  quartile).  Testing  the  sensitivity  of  the  results  to 
stage  of  contract  completion  also  served  as  an  evaluation  of  the  timeliness  of  the  methods 
and  provided  data  on  which  methods  performed  the  best  during  the  early  stages  of 
contract  completion.  To  compare  the  performance  of  the  EAC  methods  in  each  stage  of 
contract  completion,  the  data  were  divided  into  four  subsets  according  to  the  quartiles  just 
defined.  The  MAPE  and  SDAPE  were  then  calculated  for  each  EAC  method  within  each 
of  these  data  subsets.  T  he  combined  score  was  then  calculated  for  each  EAC  method 

within  each  quartile,  and  the  method  with  the  lowest  score  in  each  quartile  was  classed  as 
the  best  performing  method. 

Sensitivity  of  Results  to  Type  of  System.  Sensitivity  of  the  results  to  nine  categories 
of  system  type  was  tested.  These  categories  included:  Aircraft  (nine  contracts);  Armored 
Vehicles  (two  contracts);  Ballistic  Missiles  (18  contracts);  Electronic  Warfare  Systems 
(nine  contracts);  Helicopters  ( 1 6  contracts);  Other  Missiles  (19  contracts);  Satellites 
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(three  contracts);  Ships  (eight  contracts);  and,  Torpedoes  (four  contracts).  To  compare 
the  performance  of  the  EAC  methods  according  to  system  type,  the  data  were  divided  into 
nine  subsets  according  to  each  of  the  system  types  just  defined.  Next,  the  MAPE  and 
SDAPE  were  calculated  for  each  EAC  method  within  each  of  these  data  subsets.  Then 
the  combined  score  was  calculated  for  each  EAC  method  within  each  category,  and  the 
method  with  the  lowest  score  in  each  category  was  classed  as  the  best  performing 
method.  In  addition  to  evaluating  the  overall  performance  of  the  methods  for  each  system 
type,  the  sensitivity  of  each  system  type  result  to  stage  of  contract  completion  was  also 
tested  using  the  procedure  previously  described. 

Sensitivity  of  Results  to  Program  Phase.  Sensitivity  of  the  results  to  three  categories 
of  program  phase  was  tested.  These  categories  included:  Development  (24  contracts); 
Production  (53  contracts);  and.  Other  (1 1  contracts),  which  included  those  contracts  that 
could  not  be  categorized  in  either  of  the  first  two  categories.  To  compare  the 
performance  of  the  EAC  methods  according  to  program  phase,  the  data  were  divided  into 
three  subsets  according  to  each  the  phases  just  defined.  Next,  the  MAPE  and  SDAPE 
were  calculated  for  each  EAC  method  within  each  of  these  data  subsets.  Then  the 
combined  score  was  calculated  for  each  method  within  each  category,  and  the  method 
with  the  lowest  score  in  each  category  was  classed  as  the  best  performing  method.  In 
addition  to  evaluating  the  overall  performance  of  the  methods  for  each  program  phase, 
the  sensitivity  of  each  program  phase  result  to  stage  of  contract  completion  was  also 
tested  using  the  procedure  previously  described. 
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Sensitivity  of  Results  to  Contract  Type.  Sensitivity  of  the  results  to  three  categories 
of  contract  type  was  tested.  These  categories  included:  Cost  Plus  (21  contracts);  Fixed 
Price  (66  contracts);  and.  Mixed  (one  contract),  which  included  one  contract  that  had  both 
cost  plus  and  fixed  price  components.  To  compare  the  performance  of  the  EAC  methods 
according  to  contract  type,  the  data  were  divided  into  three  subsets  according  to  the 
categories  just  defined.  Next,  the  MAPE  and  SDAPE  were  calculated  for  each  EAC 
method  within  each  of  these  data  subsets.  Then  the  combined  score  was  calculated  for 
each  method  within  each  category,  and  the  method  with  the  lowest  score  in  each  category 
was  classed  as  the  best  performing  method.  In  addition  to  evaluating  the  overall 
performance  of  the  methods  for  each  contract  type,  the  sensitivity  of  each  contract  type 
result  to  stage  of  contract  completion  was  also  tested  using  the  procedure  previously 
described. 

Sensitivity  of  Results  to  DoD  Service  Component.  Sensitivity  of  the  results  to  three 
categories  of  DoD  service  component  was  tested.  These  categories  included:  Army  (30 
contracts);  Air  Force  (36  contracts);  and.  Navy  (22  contracts).  To  compare  the 
performance  of  the  EAC  methods  according  to  DoD  service  component,  the  data  were 
divided  into  three  subsets  according  to  the  categories  just  defined.  Next,  the  MAPE  and 
SDAPE  were  calculated  for  each  EAC  method  within  each  of  these  data  subsets.  Then 
the  combined  score  was  calculated  for  each  method  within  each  category,  and  the  method 
with  the  lowest  score  in  each  category  was  classed  as  the  best  performing  method.  In 
addition  to  evaluating  the  overall  performance  of  the  methods  for  each  DoD  service 
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component,  the  sensitivity  of  each  service  component  result  to  stage  of  contract 
completion  was  also  tested  using  the  procedure  previously  described. 

Sensitivity  to  the  Effects  of  Inflation.  To  test  for  sensitivity  of  the  results  to  the 
effects  of  inflation,  the  test  procedures  were  run  using  both  the  actual  cost  numbers  in 
Then  Year  Dollars  (TYS)  contained  in  the  DAES  database,  and  again  after  converting  the 
numbers  to  Constant  Year  1995  Dollars  (CY95$).  This  adjustment  for  inflation  was  done 
using  raw  inflation  indices  provided  by  SAF/FMCE  (Appendix  G).  Two  different  sets  of 
inflation  indices  were  used  depending  on  the  program  phase  of  a  contract.  For 
development  contracts,  the  raw  inflation  indices  for  R&D  (3600)  appropriations  were 
used;  and,  for  production  and  other  contracts,  the  raw  inflation  indices  for  Other 
Procurement  (3080)  appropriations  were  used. 

Because  the  cost  data  included  in  the  DAES  database  were  cumulative  TYS,  the 
inflation  adjustment  process  involved  three  steps.  First,  the  cumulative  cost  data 
(BCWScum,  BCWPcum,  and  ACWPcum)  were  converted  to  non-cumulative  values  by 
subtracting  the  previous  reporting  period’s  cumulative  cost  value  from  the  current 
period’s  cumulative  cost  value.  In  the  case  of  the  first  reporting  period,  the  cumulative 
cost  value  is  also  the  period  cost  value;  therefore,  no  subtraction  was  required.  Next,  the 
non-cumulative  TYS  values  were  converted  to  non-cumulative  CY95S  by  dividing  the 
TYS  values  by  the  appropriate  raw  inflation  index  based  on  the  date  of  the  cost  report  and 
the  program  phase.  Finally,  the  cumulative  CY95S  cost  values  were  calculated  by 
summing  the  non-cumulative  CY95S  cost  values  through  each  cost  reporting  period. 
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All  of  the  index-based  EAC  methods  use  the  BAC;  so  BAC  also  had  to  be  converted 
to  CY95$.  Because  there  is  no  straightforward  procedure  to  convert  the  BAC  to  a  non- 
cumulative  number  the  simplest  approach  was  to  directly  divide  the  TY$  BAC  value  by 
the  appropriate  raw  inflation  index  based  on  the  date  of  the  cost  report  and  the  program 
phase. 

Once  all  of  the  cost  data  were  adjusted  for  inflation,  the  CY95$  data  were  entered 
into  duplicates  of  the  Microsoft®  Excel  spreadsheets  used  to  calculate  the  TY$  EACs  in 
place  of  the  original  TY$  data,  and  a  complete  set  of  CY95$  EACs  was  calculated.  The 
procedures  used  to  compare  the  non-inflation  adjusted  data  were  then  duplicated  to 
determine  the  best  performing  EAC  methods  using  the  inflation  adjusted  data. 

The  conversion  from  TY$  to  CY$  is  done  as  an  integral  part  of  the  Rayleigh  MMAE 
method.  Therefore,  evaluation  of  the  performance  of  this  method  could  only  be  done  after 
the  data  had  been  adjusted  for  inflation  (i.e.,  converted  to  CY95$).  Consequently,  this 
method  was  not  included  in  the  evaluation  using  TY$. 

Justification  of  Approach 

Although  the  use  of  traditional  statistical  techniques  such  as  the  t-test  or  ANOVA 
would  be  desirable,  the  need  for  independent  data  sets  ruled  out  the  use  of  these 
techniques.  To  provide  a  more  thorough  comparison,  two  aspects  of  an  EAC  method’s 
performance  were  evaluated,  accuracy  and  stability.  The  MAPE  was  selected  as  the 
measure  of  accuracy  based  on  its  popularity  and  its  ability  to  compare  estimates  across 
data  series  of  differing  magnitudes  (Armstrong  and  Collopy,  1992;  Makridakis  and 
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Wheelwright,  1978;  Nahmias,  1993).  The  SDAPE  was  selected  as  the  measure  of 
stability  based  on  its  common  use  as  a  measure  of  the  variability  in  a  data  series  by 
statisticians  (Devore,  1991). 

Because  each  method  was  evaluated  based  on  two  different  measures  of 
performance,  a  combined  measure  had  to  be  developed.  The  combined  measure  chosen 
was  to  rank  order  all  of  the  methods  according  to  both  performance  measures  and 
calculate  a  combined  score  based  on  the  sum  of  the  rank  orderings  for  each  method.  The 
method  with  the  lowest  combined  score  was  classed  as  the  best  performing  method  for  a 
given  data  series.  This  combined  performance  measure  was  felt  to  be  the  simplest  and 
most  meaningful  way  to  rate  a  method’s  performance  based  on  two  different  aspects  of 
performance.  Tests  for  sensitivity  to  stage  of  contract  completion,  system  type,  program 
phase,  contract  type,  DoD  service  component,  and  the  effects  of  inflation  were  also 
included  to  strengthen  the  significance  and  usefulness  of  the  overall  results. 

The  next  chapter  presents  the  results  of  the  overall  performance  of  the  EAC  methods 
and  the  sensitivity  of  these  results  to  stage  of  contract  completion,  system  type,  program 
phase,  contract  type,  DoD  service  component,  and  the  effects  of  inflation. 
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IV.  Analysis  of  Results 


Results  of  Overall  Performance 

The  results  for  the  performance  of  the  EAC  methods  with  all  contracts  included  in 
the  analysis  are  summarized  in  Table  6.  This  table  lists  the  best  performing  EAC 
methods  overall  and  by  stage  of  contract  completion,  for  both  the  inflation  adjusted 
(CY95S)  and  the  non-inflation  adjusted  (TY$)  data.  The  scores,  MAPE  values,  and 
SDAPE  values  for  all  of  the  EAC  methods  and  all  data  subsets  are  too  voluminous  to 
include  in  the  body  of  the  text  and  therefore  will  only  be  included  for  the  overall  results 
(Table  7).  The  complete  set  of  scores,  MAPE  values,  and  SDAPE  values,  for  the  overall 
results  as  well  as  the  results  for  each  of  the  data  subsets  used  in  the  sensitivity  analysis, 
have  been  included  as  Appendix  H. 


TABLE  6 

BEST  PERFORMING  EAC  METHODS  WITH  ALL  CONTRACTS  INCLUDED 

(88  CONTRACTS) 


Contract  Completion  Stage 

Overall 

0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 


Inflation  Adjusted  Data 

Composite 

Composite 

CPIeum 

CPI6 

SPI6 


Non-Inflation  Adjusted  Data 

Composite 

Composite 

Composite 

CPI6 

CPIeum?  Composite 
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The  best  performing  EAC  method  with  all  contracts  included  in  the  analysis  was  the 
index-based  method  using  the  Composite  Index  (0.2SPlCum+0.8CPICum)-  This  result  held 
for  both  the  inflation  adjusted  data  and  the  non-inflation  adjusted  data.  The  fact  that  this 
method  was  a  top  performer  in  the  first  quartile  for  the  inflation  adjusted  data,  and  in  the 
first,  second,  and  fourth  quartiles  for  the  non-inflation  adjusted  data  strengthens  the 
significance  of  this  overall  result.  The  overall  results  were  sensitive  to  stage  of  contract 
completion,  although  this  sensitivity  was  less  significant  for  the  non-inflation  adjusted 
data.  The  first  and  third  quartile  results  were  not  sensitive  to  the  effects  of  inflation; 
however,  the  second  and  fourth  quartile  results  were. 

TABLE  7 

RESULTS  FOR  OVERALL  PERFORMANCE 


Inflation  Adjusted  Data  Non-Inflation  Adiusted  Data 


EAC  Method 

Score* 

MAPE 

CPI3 

10 

13.0% 

CPI6 

5 

11.7% 

CPIcum 

4 

11.6% 

SPI3 

16 

15.2% 

SP16 

7 

13.0% 

SPIcum 

9 

13.0% 

SCI3 

18 

19.4% 

SCI6 

13 

14.1% 

SCIcum 

13 

13.6% 

CPI6xSPlCum 

9 

13.6% 

Composite 

2 

1 1 .3% 

Rayleigh  Least  Squares 

23 

237.0% 

Rayleigh  Three  Group 

23 

300.8% 

Rayleigh  MMAE 

20 

235.6% 

Beta 

26 

>500% 

SDAPE 

Score 

MAPE 

SDAPE 

19.3% 

14 

12.9% 

19.3% 

14.4% 

4 

1 1 .4% 

14.6% 

14.4% 

6 

11.6% 

14.9% 

24.2% 

17 

14.7% 

23.1% 

17.0% 

8 

12.5% 

16.5% 

19.0% 

10 

12.5% 

17.7% 

92.3% 

19 

19.0% 

94.9% 

19.7% 

14 

13.4% 

18.9% 

20.0% 

12 

12.9% 

18.6% 

18.0% 

10 

12.9% 

17.1% 

14.0% 

2 

11.2% 

14.5% 

>500% 

21 

206.6% 

>500% 

>500% 

23 

449.9% 

>500% 

494.7% 

N/A 

N/A 

N/A 

>500% 

25 

>500% 

>500% 

*  Score  calculated  by  rank  ordering  EAC  methods  by  increasing  value  of  M  APE  and 
SDAPE  and  summing  rankings.  Method  with  lowest  score  is  best  performing  method. 
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Table  7  has  been  included  to  allow  a  more  detailed  discussion  and  analysis  of  the 
overall  results.  In  addition,  the  combined  scores  for  the  overall  results  are  shown 
graphically  in  Figure  1 .  It  can  be  seen  from  an  examination  of  the  MAPE  and  SDAPE 
values  in  Table  7,  and  of  Figure  1,  that  the  index-based  EAC  methods  overall  performed 
significantly  better  than  any  of  the  nonlinear  regression-based  methods.  A  comparison  of 
the  results  for  the  inflation  adjusted  data  highlights  this  point.  The  worst  performing 
index-based  method  was  the  CPI3  with  a  MAPE  of  19.4  percent  and  an  SDAPE  of  92.3 
percent.  On  the  other  hand,  the  best  performing  nonlinear  regression-based  method  was 
the  Rayleigh  MMAE  method  with  a  MAPE  of  235.6  percent  and  an  SDAPE  of  494.7 
percent.  A  similar  observation  can  be  made  for  the  non-inflation  adjusted  data.  These 
results  support  the  hypothesis  that  the  complex  nonlinear  regression-based  EAC  methods 
do  not  perform  better  than  the  simple  index-based  methods,  as  stated  in  Chapter  I. 


Figure  1 .  Relative  Performance  of  EAC  Methods 
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The  modified  version  of  the  SCI  (CPl6xSPIcum),  while  not  the  top  performing,  method 
overall,  did  perform  better  than  the  index-based  methods  using  CPI3,  SPI3,  SCI3,  SCI6, 
and  SCIcum-  For  the  remaining  index-based  methods  two  observations  can  be  made. 

First,  the  index -based  methods  using  the  six  month  and  cumulative  indices  performed 
better  than  the  index-based  methods  using  the  three  month  indices.  Second,  the  CPI  and 
SPI-based  methods  generally  outperformed  the  SCI  with  the  exception  of  the  SPI3  which 
was  outperformed  by  both  the  SCI6  and  SCICUm-  These  observations  hold  for  both  the 
inflation  adjusted  and  non-inflation  adjusted  data.  Both  of  these  trends  are  highlighted 
graphically  in  Figure  2. 


Figure  2.  Comparison  of  Three  Month,  Six  Month,  and  Cumulative  Indices 


With  the  exception  of  the  Rayleigh  three  group  linear  regression  method  and  the 
index-based  method  using  CPIcum,  the  non-inflation  adjusted  EACs  were  slightly  more 
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accurate  (as  measured  by  the  MAPE)  than  those  calculated  using  the  inflation  adjusted 
data.  The  methodology  used  to  adjust  the  BAC  for  inflation  may,  however,  have  an  effect 
on  the  accuracy  of  the  inflation  adjusted  EACs  and  could  be  the  cause  of  this  effect.  A 
comparison  of  the  accuracy  of  the  inflation  adjusted  data  and  the  non-inflation  adjusted 
data  for  the  index -based  EAC  methods  is  shown  graphically  in  Figure  3.  The  nonlinear 
regression-based  EAC  methods  were  not  included  in  this  figure  because  the  large 
differential  in  MAPE  values  made  the  inclusion  of  these  results  on  the  same  graph  with 
the  index-based  EAC  methods  difficult.  The  difference  in  accuracy  for  the  nonlinear 
regression-based  EAC  methods  can  be  seen  from  an  examination  of  the  MAPE  values  in 
Table  7.  While  the  use  of  non-inflation  adjusted  data  produces  slightly  more  accurate 
EACs  overall,  an  analysis  of  the  detailed  results  for  the  various  data  subsets  included  in 
Appendix  H,  shows  that  this  observation  does  not  always  hold. 


Figure  3.  Comparison  of  Index-Based  EAC  Method  Accuracy  Using  Inflation  Adjusted 

Data  and  Non-Inflation  Adjusted  Data 


59 


The  remainder  of  this  chapter  is  devoted  to  presenting  the  results  of  the  sensitivity 
analysis,  followed  by  a  comparison  of  the  results  of  this  study  to  previous  comparative 
EAC  studies  and  some  concluding  remarks. 

Sensitivity  of  Results  to  Type  of  System 

The  best  performing  EAC  methods  for  each  of  the  nine  System  Types  (Aircraft, 
Armored  V ehicles.  Ballistic  Missiles,  Electronic  Warfare,  Helicopter,  Other  Missiles, 
Satellites,  Ships,  and  Torpedoes)  are  shown  in  Tables  8  through  16,  and  each  table  is 
followed  by  a  brief  discussion  of  the  results.  A  review  of  these  tables  shows  that  the 
overall  results  were  sensitive  to  the  System  Type. 


TABLE  8 

BEST  PERFORMING  EAC  METHODS  FOR  AIRCRAFT  CONTRACTS 

(9  CONTRACTS) 


Contract  ComDletion  Stage 

Inflation  Adjusted  Data 

Non-Inflation  Adjusted  Data 

Overall 

SPI6 

CPI6xSPICum 

0%  to  25% 

CPI6xSPIcum 

CPI6xSPIcum 

>25%  to  50% 

CPI3,  CPI6,  CPIcum 

CPI3,  CPI6 

>50%  to  75% 

CPIcurrb  SCIcum, 
Composite 

SCIcum 

>75%  to  100% 

SPI6 

CPIcum?  SPI<j,  Composite 

Overall,  the  best  performing  method  for  the  Aircraft  Contracts  depended  on  whether 
the  data  were  adjusted  for  inflation.  For  the  inflation  adjusted  data,  the  best  performing 
EAC  method  was  the  index-based  method  using  SPL.  For  the  non-inflation  adjusted 
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data,  the  best  performing  method  was  the  index-based  method  using  CPl6xSPIcum.  In 
general,  these  results  were  sensitive  to  both  the  stage  of  contract  completion  and  the 
effects  of  inflation. 


TABLE  9 

BEST  PERFORMING  EAC  METHODS  FOR  ARMORED  VEHICLE  CONTRACTS 

(2  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

CPIfcXSPIcum 

CPI^xSPIcum 

0%  to  25% 

SPI6 

SPI6 

>25%  to  50% 

CPI3,  CPI6,  SCIcum, 
CPI6xSPIcum 

CPI3,  CPI6xSPIcum 

>50%  to  75% 

CPI3 

CPI3 

>75%  to  100% 

SCI3 

SPI3 

Overall,  the  best  performing  method  for  the  Armored  Vehicle  contracts  was  the 
index-based  method  using  CPI6xSPIcum.  This  overall  result  was  not  sensitive  to  the 
effects  of  inflation.  The  results  for  the  armored  vehicle  data  were  sensitive  to  stage  of 
contract  completion.  Sensitivity  of  the  results  to  the  effects  of  inflation  was  dependent  on 
stage  of  contract  completion.  Unfortunately  the  small  number  of  contracts  in  this 
category  limits  the  ability  to  generalize  these  results. 
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TABLE  10 


BEST  PERFORMING  EAC  METHODS  FOR  BALLISTIC  MISSILE  CONTRACTS 

(18  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adjusted  Data 

Non-Inflation  Adjusted  Data 

Overall 

Composite 

Composite 

0%  to  25% 

Composite 

Composite 

>25%  to  50% 

CPI3,  CPI6,  SCI6 

CPI6,  SCI6 

>50%  to  75% 

CPI3 

CPI3 

>75%  to  100% 

SCI3 

SCI3 

Overall,  the  best  performing  method  for  the  Ballistic  Missile  contracts  was  the  index- 
based  method  using  the  Composite  Index.  This  overall  result  was  not  sensitive  to  the 
effects  of  inflation.  The  results  for  the  Ballistic  Missile  data  were  sensitive  to  stage  of 
contract  completion.  Generally  these  results  were  not  sensitive  to  the  effects  of  inflation. 


TABLE  1 1 

BEST  PERFORMING  EAC  METHODS  FOR  ELECTRONIC  WARFARE  SYSTEM 

CONTRACTS 
(9  CONTRACTS) 

Contract  Completion  Staee 

Inflation  Adjusted  Data 

Non-Inflation  Adjusted  Data 

Overall 

0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 

SPIcum 

SPIcum 

SPIcum 

CPI6 

SPI3,  SPIcum 

SPIcum 

SPIcum 

SPIcum 

CPI6 

CPIcum,  Composite 
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Overall,  the  best  performing  method  for  the  Electronic  Warfare  System  contracts  was 
the  index-based  method  using  SPIcum.  This  overall  result  was  not  sensitive  to  the  effects 
of  inflation.  Stage  of  contract  completion  only  affected  the  results  in  the  last  two  stages 
of  contract  completion.  Only  the  result  for  the  last  stage  of  contract  completion  was 
sensitive  to  the  effects  of  inflation. 


TABLE  12 

BEST  PERFORMING  EAC  METHODS  FOR  HELICOPTER  CONTRACTS 

(16  CONTRACTS) 


Contract  Completion  Stage 

Overall 
0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 


Inflation  Adjusted  Data 

Composite 
Composite 
Composite 
CPIcum,  Composite 
Beta 


Non-Inflation  Adjusted  Data 

Composite 

Composite 

SPIcum 

CPICum,  Composite 
SPI*  SPIcum 


Overall,  the  best  performing  method  for  the  Helicopter  contracts  was  the  index-based 
method  using  the  Composite  Index.  This  overall  result  was  not  sensitive  to  the  effects  of 
inflation.  The  results  were  sensitive  to  stage  of  contract  completion.  In  two  of  the  four 
stages  of  contract  completion  the  results  were  sensitive  to  the  effects  of  inflation.  In  fact, 
in  the  final  stage  of  contract  completion  using  the  inflation  adjusted  data,  the  nonlinear 
Beta  curve  method  was  the  best  performing  method.  Given  the  reasonably  large  sample 
size  (16  contracts),  this  is  a  compelling  result.  Unfortunately,  because  the  method  only 
performs  well  during  the  final  stage  of  contract  completion  its  usefulness  is  limited.  And, 
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in  fact,  a  review  of  the  detailed  Helicopter  contract  results  in  Appendix  H  indicates  that  it 
is  the  worst  performing  method  during  the  first  two  stages  of  contract  completion  further 
limiting  the  usefulness  of  the  method. 


TABLE  13 

BEST  PERFORMING  EAC  METHODS  FOR  OTHER  MISSILE  CONTRACTS 

(19  CONTRACTS) 


Contract  Completion  Stage 

Overall 
0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 


Inflation  Adjusted  Data 

Composite 

SPIcum 

CPI3 

CPI6xSPIcum 

SPI6 


Non-Inflation  Adjusted  Data 

Composite 

SPIcum 

CPI3 

CPI6 

CPIcum 


Overall,  the  best  performing  method  for  the  Other  Missile  contracts  was  the  index- 
based  method  using  the  Composite  Index.  This  overall  result  was  not  sensitive  to  the 
effects  of  inflation.  The  results  were  sensitive  to  stage  of  contract  completion.  During 
the  first  half  of  the  contracts  the  results  were  not  sensitive  to  the  effects  of  inflation. 
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TABLE  14 


BEST  PERFORMING  EAC  METHODS  FOR  SATELLITE  CONTRACTS 

(3  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

Composite 

SPI6,  SPIcum 

0%  to  25% 

CPI6,  CPICUm 

SPI6 

>25%  to  50% 

CPIcum 

Composite 

>50%  to  75% 

SCIcum 

SPIcum 

>75%  to  100% 

Beta 

SPI6 

Overall,  the  best  performing  method  for  the  Satellite  contracts  depended  on  the 
effects  of  inflation.  For  the  inflation  adjusted  data,  the  best  performing  method  was  the 
index-based  method  using  the  Composite  Index.  For  the  non-inflation  adjusted  data,  two 
methods  were  the  top  performers.  They  were  the  index-based  methods  using  SPL  and 
SPIcum.  The  results  were  sensitive  to  stage  of  contract  completion,  and  to  the  effects  of 
inflation.  Once  again  the  nonlinear  Beta  curve  method  was  the  best  performing  method 
in  the  final  stage  of  contract  completion  for  the  inflation  adjusted  data.  The  ability  to 
generalize  these  results  is  limited  by  the  small  number  of  contracts  in  this  category. 


65 


TABLE  15 


BEST  PERFORMING  EAC  METHODS  FOR  SHIP  CONTRACTS 

(8  CONTRACTS) 


Contract  Completion  Stage 

Overall 

0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 


Inflation  Adjusted  Data 

Composite 

Composite 

Composite 

CPI6 

SPL,  SPIcum,  Composite 


Non-Inflation  Adjusted  Data 

Composite 
Composite 
CPIcum,  Composite 
CPI6 

CPIcum,  Composite 


Overall,  the  best  performing  method  for  the  Ship  contracts  was  the  index -based 
method  using  the  Composite  Index.  This  result  was  not  sensitive  to  the  effects  of 
inflation.  In  fact,  the  Composite  Index  was  a  top  performer  in  all  but  one  stage  of 
contract  completion,  demonstrating  limited  sensitivity  to  both  factors.  The  only  time  the 
Composite  Index  was  not  a  top  performer  was  in  the  third  quartile  for  both  the  inflation 
adjusted  and  non-inflation  adjusted  data. 
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TABLE  16 


BEST  PERFORMING  EAC  METHODS  FOR  TORPEDO  CONTRACTS 

(4  CONTRACTS) 


Contract  Completion  Stage 

Overall 

0%  to  25% 

>25%  to  50% 

>50%  to  75% 

>75%  to  100% 


Inflation  Adjusted  Data 

CPI^,  CPIcum,  SPIcum, 

Composite 

SPIcum 

CPI3,  SPI  cum?  SCIcum 
CPI3,  CPIcum,  SCI3 
SPI6 


Non-Inflation  Adjusted  Data 

CPI6 

SPIcum 
CPI3,  SCIcum 
CPI3,  SCI3,  SCI  cum 

spi6 


Overall,  the  best  performing  method  for  the  Torpedo  contracts  depended  on  the 
effects  of  inflation.  The  index-based  method  using  CPL  performed  well  in  both  cases; 
however,  three  other  index-based  methods  were  also  top  performers  using  the  inflation 
adjusted  data.  They  were:  CPICUm;  SPICUm;  and,  the  Composite  index.  Generally,  the 
results  for  the  Torpedo  contracts  were  sensitive  to  the  effects  of  inflation  and  stage  of 
contract  completion.  The  small  number  of  contracts  limits  the  ability  to  generalize  these 
results. 

Sensitivity  of  Results  to  Program  Phase 

The  best  performing  EAC  methods  for  each  of  the  three  categories  of  Program  Phase 
(Development,  Production,  and  Other)  are  shown  in  Tables  17  through  19.  Each  table  is 
followed  by  a  brief  discussion  of  the  results.  A  review  of  these  tables  shows  that  the 
overall  results  were  sensitive  to  Program  Phase. 
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TABLE  17 


BEST  PERFORMING  EAC  METHODS  FOR  DEVELOPMENT  CONTRACTS 

(24  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adjusted  Data 

Non-Inflation  Adjusted  Data 

Overall 

SPI6,  SPICum 

CPl6xSPICUm 

0%  to  25% 

SPIcum 

SPIcum 

>25%  to  50% 

SPI3 

SCI3 

>50%  to  75% 

CPI6xSPIcum 

CPI6xSPIcum 

>75%  to  100% 

SP  Icum 

CPIcum,  Composite 

Overall,  the  best  performing  method  for  the  Development  contracts  depended  on  the 
effects  of  inflation.  For  the  inflation  adjusted  data,  the  two  index-based  methods  using 
SPI6,  and  SPICum  were  the  top  performing  methods.  For  the  non-inflation  adjusted  data, 
the  index-based  method  using  CPI6xSPIcum  was  the  top  performer.  The  results  were 
sensitive  to  stage  of  contract  completion.  However,  the  only  results  sensitive  to  the 
effects  of  inflation  were  the  overall  result  and  the  result  for  fourth  quartile. 
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TABLE  18 


BEST  PERFORMING  EAC  METHODS  FOR  PRODUCTION  CONTRACTS 

(53  CONTRACTS) 


Contract  Completion  Stage  Inflation  Adjusted  Data  Non-Inflation  Adjusted  Data 


Overall 

0%  to  25% 
>25%  to  50% 
>50%  to  75% 

>75%  to  100% 


Composite 
Composite 
CPI6,  Composite 
CPI6,  CPIcum, 
Composite 
CPIcum 


Composite 
Composite 
Composite 
CPI6,  CPICum 

CPIcUm,  Composite 


Overall,  the  best  performing  method  for  the  Production  contracts  was  the  index- 
based  method  using  the  Composite  Index.  This  result  was  not  sensitive  to  the  effects  of 
inflation.  In  fact,  the  Composite  Index  was  a  top  performer  in  all  but  one  stage  of 
contract  completion  for  the  inflation  adjusted  data  (the  fourth  quartile),  and  the  non¬ 
inflation  adjusted  data  (the  third  quartile),  demonstrating  limited  sensitivity  to  both 
factors. 


TABLE  19 

BEST  PERFORMING  EAC  METHODS  FOR  OTHER  CONTRACTS 

(1 1  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

CPI6 

SCIcum?  CPI^xSPIcum 

0%  to  25% 

SCI6 

SCI6 

>25%  to  50% 

CPI6.  CPIcum,  SCIcum 

CPI6,  CPIcum?  SCICum 

>50%  to  75% 

CPI3 

CPIcum 

>75%  to  100% 

Beta 

Beta 
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Overall,  the  best  performing  method  for  the  Other  contracts  depended  on  the  effects 
of  inflation.  For  the  inflation  adjusted  data,  the  index-based  method  using  CPI6,  was  the 
top  performing  method.  For  the  non-inflation  adjusted  data,  the  index -based  methods 
using  SCIcum  and  CPl6xSPIcum  were  the  top  performers.  The  results  were  sensitive  to 
stage  of  contract  completion.  The  quartile  results  were  generally  not  sensitive  to  the 
effects  of  inflation  except  in  the  third  quartile.  Again,  the  Beta  method  was  the  best 
performing  method  in  the  fourth  quartile.  In  this  case,  it  was  the  best  performing  method 
for  both  the  inflation  adjusted  and  the  non-inflation  adjusted  data.  As  in  the  other  cases,  a 
review  of  the  detailed  results  in  Appendix  H  indicates  that  this  method  was  a  poor 
performer  in  the  early  stages  of  contract  completion,  limiting  its  usefulness. 

Sensitivity  of  Results  to  Contract  Type 

The  best  performing  EAC  methods  for  each  of  the  three  categories  of  Contract  Type 
(Cost  Plus,  Fixed  Price,  and  Mixed)  are  shown  in  Tables  20  through  22.  Each  table  is 
followed  by  a  brief  discussion  of  the  results.  A  review  of  these  tables  shows  that  the 
overall  results  were  sensitive  to  Contract  Type. 
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TABLE  20 


BEST  PERFORMING  EAC  METHODS  FOR  COST  PLUS  CONTRACTS 

(21  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

SPI6 

CPI6xSPIcum 

0%  to  25% 

CPI<jxSPIcum 

SPIcum?  SCI6,  CPI^X SPIcum 

>25%  to  50% 

SPIcum 

SPIcum?  CPIs*  SPIcum 

>50%  to  75% 

SCIcum 

SCIcum 

>75%  to  100% 

SPI3 

CPIcum 

Overall,  the  best  performing  method  for  the  Cost  Plus  contracts  depended  on  the 
effects  of  inflation.  For  the  inflation  adjusted  data,  the  index-based  method  using  SPL, 
was  the  top  performing  method.  For  the  non-inflation  adjusted  data,  the  index -based 
method  using  CPI6xSPIcum  was  the  top  performer.  The  results  were  sensitive  to  stage  of 
contract  completion  and  the  effects  of  inflation  except  the  third  quartile  result. 


TABLE  21 


BEST  PERFORMING  EAC  METHODS  FOR  FIXED  PRICE  CONTRACTS 

(66  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

Composite 

Composite 

0%  to  25% 

Composite 

Composite 

>25%  to  50% 

CPI3,  CPI6 

CPI6 

>50%  to  75% 

CPI6 

CPI6 

>75%  to  100% 

CPIcum,  Composite 

CPIcum,  Composite 
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Overall,  the  best  performing  method  for  the  Fixed  Price  contracts  was  the  index- 
based  method  using  the  Composite  Index.  This  result  was  not  sensitive  to  the  effects  of 
inflation.  The  results  were  sensitive  to  stage  of  contract  completion;  however,  they 
generally  were  not  sensitive  to  the  effects  of  inflation  except  for  the  addition  of  the  index- 
based  method  using  CPI3  to  the  third  quartile  result  for  the  inflation  adjusted  data. 

TABLE  22 


BEST  PERFORMING  EAC  METHODS  FOR  MIXED  TYPE  CONTRACTS 

(1  CONTRACT) 


Contract  Completion  Stage 

Inflation  Adjusted  Data 

Non-Inflation  Adjusted  Data 

Overall 

SPI3 

SPI6,  SPIcum 

0%  to  25% 

SPI3,  SPIcum 

SPIcum 

>25%  to  50% 

CPl6><SPIcum 

CPU*  SPICum 

>50%  to  75% 

Rayleigh  Linear 
Regression 

SPI3 

>75%  to  100% 

SCI6 

SCI6 

Overall,  the  best  performing  method  for  the  single  Mixed  Type  contract  depended  on 
the  effects  of  inflation.  For  the  inflation  adjusted  data  the  best  performing  method  was 
the  index -based  method  using  SPI3.  For  the  non-inflation  adjusted  data,  the  best 
performing  methods  were  the  index-based  methods  using  SPI6  and  SPICUm-  The  results 
were  sensitive  to  stage  of  contract  completion  and  to  the  effects  of  inflation  except  in  the 
second  and  fourth  quartiles.  It  is  interesting  to  note  that  in  the  third  quartile  for  the 
inflation  adjusted  data  the  Rayliegh  Linear  Regression  technique  is  the  best  performing 
method.  However,  a  review  of  the  detailed  results  in  Appendix  H  shows  that  it  is  a  poor 
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performer  in  all  of  the  remaining  quartiles  and  overall.  Because  there  was  only  one 
contract  in  this  category,  the  ability  to  generalize  these  results  is  severely  limited. 

Sensitivity  of  Results  to  DoD  Service  Component 

The  best  performing  EAC  methods  for  each  of  the  three  categories  of  DoD  service 
component  (Army,  Air  Force,  and  Navy)  are  shown  in  Tables  23  through  25.  Each  table 
is  followed  by  a  brief  discussion  of  the  results.  A  review  of  these  tables  shows  that  the 
overall  results  were  sensitive  to  DoD  Service  Component. 


TABLE  23 

BEST  PERFORMING  EAC  METHODS  FOR  ARMY  CONTRACTS 

(30  CONTRACTS) 


Contract  Completion  Stage 

Inflation  Adiusted  Data 

Non-Inflation  Adiusted  Data 

Overall 

Composite 

Composite 

0%  to  25% 

Composite 

Composite 

>25%  to  50% 

CPL,  Composite 

CPI6 

>50%  to  75% 

CPI6 

CPI6 

>75%  to  100% 

SPIcum,  Composite 

CPIcum,  SPI6,  Composite 

Overall,  the  best  performing  method  for  the  Army  contracts  was  the  index-based 
method  using  the  Composite  Index.  This  result  was  not  sensitive  to  the  effects  of 
inflation.  The  results  were  sensitive  to  stage  of  contract  completion.  The  quartile  results 
were  sensitive  to  the  effects  of  inflation  except  in  the  first  and  third  quartiles. 


73 


TABLE  24 


BEST  PERFORMING  EAC  METHODS  FOR  AIR  FORCE  CONTRACTS 

(36  CONTRACTS) 


Contract  Completion  Stage  Inflation  Adjusted  Data  Non-Inflation  Adjusted  Data 


Overall 

0%  to  25% 
>25%  to  50% 
>50%  to  75% 
>75%  to  100% 


Composite 
Composite 
CPI3,  CPI6 
CPI3,  CPI6 
SPI3 


Composite 

Composite 

CPI6 

CPIs 

CPIcum 


Overall,  the  best  performing  method  for  the  Air  Force  contracts  was  the  index-based 
method  using  the  Composite  Index.  This  result  was  not  sensitive  to  the  effects  of 
inflation.  The  results  were  sensitive  to  stage  of  contract  completion.  The  quartile  results 
were  sensitive  to  the  effects  of  inflation  except  in  the  first  quartile. 

TABLE  25 

BEST  PERFORMING  EAC  METHODS  FOR  NAVY  CONTRACTS 

(22  CONTRACTS) 


Contract  Completion  Stage  Inflation  Adjusted  Data  Non-Inflation  Adjusted  Data 


Overall 
0%  to  25% 
>25%  to  50% 

>50%  to  75% 
>75%  to  100% 


CPI(,,  Composite 
Composite 
CPI3,  CPI6 


CPIcum 

SPI6 


CPI6 

CPI6 

CPI6,CPIcum, 

Composite 


SPI 


cum? 


SCI 


cum? 


CPIcum 

SPIcum,  Composite 


Overall,  the  best  performing  method  for  the  Navy  contracts  depended  on  the  effects 
of  inflation.  For  the  inflation  adjusted  data,  the  index-based  methods  using  CPI6  and  the 
Composite  Index  were  the  top  performing  methods.  For  the  non-inflation  adjusted  data, 
the  index-based  method  using  CPI6  was  the  top  performing  method.  The  results  were 
sensitive  to  stage  of  contract  completion.  Only  the  quartile  result  for  the  third  quartile 

was  not  sensitive  to  the  effects  of  inflation. 

Comparison  of  Results  to  Previous  Research 

The  comparison  of  the  results  of  this  study  to  the  results  of  previous  comparative 
EAC  research  is  difficult  because  the  diversity  of  the  data  included  m  this  study  is  much 
greater  than  was  used  in  the  studies  that  were  reviewed.  There  are,  however,  two  sets  of 
results  from  the  sensitivity  analyses  that  are  comparable  to  two  of  the  previous  studies. 

First,  the  sensitivity  results  for  all  Navy  contracts  can  be  compared  to  the  results  of 
the  Covach,  et  al.  study  (1981).  Recall  from  Chapter  II  that  the  study,  using  data  from  six 
Navy  contracts  (five  Development  and  one  Production),  found  the  index-based  methods 
using  CPI6  and  CPI  )2  to  be  the  best  performing  methods  overall  (Covach,  et  al.,  1981). 
The  index-based  method  using  CPI  12  was  not  tested  in  this  study.  However,  the  method 
using  CPI6  was  tested,  and  found  to  be  the  best  performing  method  overall  for  the  22 
Navy  contracts  analyzed  in  this  study,  which  is  supported  by  the  results  from  the  Covach, 
et  al.  study.  This  comparison  was  made  using  the  results  from  the  non-inflation  adjusted 
data  because  adjustment  for  inflation  was  not  done  in  the  Covach  et  al.  study.  A 
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comparison  of  the  quartile  results  was  not  possible  due  to  the  differing  stages  of  contract 
completion  used  in  the  two  studies. 

The  second  case  where  a  comparison  was  possible  was  the  Land  and  Preston  study 
(1980).  Recall  from  Chapter  II  that  the  study,  using  data  from  20  Air  Force  aircraft 
programs,  found  the  index-based  method  using  CPICum  to  be  the  best  performing  method 
overall  (Land  and  Preston,  1980).  Because  the  sensitivity  results  for  aircraft  contracts  in 
this  study  included  contracts  from  all  three  DoD  Service  Components,  it  was  concluded 
that  the  results  would  not  be  comparable.  A  comparison  to  the  sensitivity  results  from  the 
overall  Air  Force  contracts  seemed  more  appropriate.  In  this  study,  the  best  performing 
method  for  the  overall  Air  Force  contracts  was  the  index-based  method  using  the 
Composite  Index.  This  method  was  not,  however,  tested  by  Land  and  Preston.  On  the 
other  hand,  if  the  index-based  method  using  the  Composite  Index  was  excluded  from  the 
results  of  this  study,  the  top  performing  method  would  be  the  index-based  method  using 
CPI6,  which  is  supported  by  the  results  of  Land  and  Preston’s  research.  Because  Land 
and  Preston  did  not  analyze  their  results  based  on  stage  of  contract  completion,  a 
comparison  of  the  quartile  results  was  not  possible. 

A  comparison  to  the  remaining  comparative  EAC  studies  was  concluded  to  be 
meaningless.  First,  Heydinger  only  used  data  from  a  single  Air  Force  Development 
contract  making  the  results  from  that  study  relatively  insignificant  (1977).  Second,  the 
Bright  and  Howard  study  only  used  data  from  1 1  Army  R&D  contracts  (1981).  Because 
the  data  in  this  study  were  only  analyzed  using  a  category  which  included  all  Army 
contracts,  and  did  not  analyze  the  data  by  program  phase,  a  comparison  was  concluded  to 
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be  inappropriate.  The  final  study  by  Reidel  and  Chance,  which  included  data  from  64  Air 
Force  contracts,  only  presented  quartile  and  overall  results  based  on  system  type  (Aircraft, 
Avionics,  and  Engines)  and  program  phase  (Development  and  Production),  no  overall 
result  for  all  contracts  was  included  in  the  study  (1989).  Because  the  results  in  this  study 
were  only  categorized  at  the  overall  Air  Force  level,  a  comparison  was  concluded  to  be 
inappropriate. 

Summary' 

With  the  exception  of  the  Beta  method  in  three  cases,  and  the  Rayleigh  Linear 
Regression  Method  in  one  case,  the  index-based  methods  were  always  the  top  performing 
methods.  Other  than  the  exceptions  noted  the  results  for  the  nonlinear  regression  based 
methods  were  significantly  worse  than  the  results  for  the  index-based  methods.  The 
original  hypothesis  that  the  complex  nonlinear  regression-based  EAC  methods  do  not 
perform  better  them  the  simple  index-based  EAC  methods  was  supported  by  the  results  of 
this  study. 

While  no  single  index -based  method  was  always  the  top  performer  after  the  data 
were  broken  down  for  sensitivity  analysis,  the  overall  and  early  quartile  performance  of 
the  method  using  the  Composite  Index,  for  all  contracts  and  in  a  number  of  the  sensitivity 
analyses,  is  notable.  Generally,  however,  the  stage  of  contract  completion,  system  type, 
program  phase,  contract  type,  DoD  service  component,  and  inflation  effects  play  a  major 
role  in  determining  which  EAC  method  is  the  top  performer.  Based  on  the  results  of  this 
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study,  it  is  clear  that  all  of  these  factors  should  be  considered  when  selecting  an  EAC 
method  for  a  particular  program. 
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Chapter  V.  Conclusions  and  Recommendations 


The  purpose  of  this  study  was  to  determine  if  the  complex  nonlinear  regression-based 
EAC  methods  performed  better  than  the  simpler  index-based  EAC  methods.  This  chapter 
summarizes  the  research  conducted  in  this  study  by  first  reviewing  the  hypothesis.  Next 
the  analysis  methods  and  results  are  reviewed.  Finally,  the  chapter  concludes  with  a 
discussion  of  the  findings  and  recommendations  for  further  research. 

Review  of  Hypothesis 

The  primary  hypothesis  tested  in  this  research,  as  stated  in  Chapter  I,  was: 

EACs  calculated  using  complex  nonlinear  regression-based  methods  are  not  more 
accurate  or  stable  than  EACs  calculated  using  simple  index -based  methods. 

In  addition  to  testing  this  hypothesis,  the  study  also  investigated  the  sensitivity  of  the 
results  to  such  moderator  variables  as  stage  of  contract  completion,  system  type,  program 
phase,  contract  type,  DoD  service  component,  and  inflation  effects. 

Review  of  Analysis  Methodology  and  Results 

This  hypothesis  was  tested  using  a  sample  of  88  contracts  extracted  from  the  DAES 
database.  These  contracts  were  assumed  to  be  a  representative  sample  of  defense 
contracts  from  all  three  DOD  service  components.  The  original  database  included  over 
541  contracts;  however,  the  data  needed  to  be  screened  to  eliminate  contracts  with 
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insufficient  or  problematic  data.  One  of  the  major  requirements  was  the  need  for  an 
objective  measure  of  the  CAC  for  each  contract.  This  was  required  as  a  comparison  point 
to  evaluate  the  performance  of  the  EAC  methods.  The  ACWPCUm  as  of  the  final  cost 
report  was  chosen  as  the  most  representative  measure  of  a  contract’s  CAC,  and  any 
contracts  without  data  beyond  the  90  percent  completion  point  were  eliminated  from  the 
analysis.  In  addition,  contracts  without  data  prior  to  the  10  percent  completion  point, 
contracts  without  a  definitive  start  date,  and  contracts  with  an  unstable  baseline  (i.e., 
contracts  that  went  OTB)  were  eliminated  from  the  analysis.  Because  only  contracts  with 
a  stable  baseline  were  included  in  the  analysis,  it  is  important  to  note  that  the  results  of 
this  study  are  only  applicable  to  future  contracts  with  a  stable  baseline. 

Fifteen  different  EAC  methods  were  tested  in  this  study,  including  1 1  index-based 
methods  (CPI3,  CPU,  CPICum,  SPI3,  SPU,  SPICUm,  SCI3,  SCU,  SCIcum,  CPUxSPIcum,  and 
0.2SPIcum+0.8CPIcum)  and  four  nonlinear  regression-based  methods  (Rayleigh  Least 
Squares  Linear  Regression,  Rayleigh  Three  Group  Linear  Regression,  Rayleigh  MMAE, 
and  Beta). 

The  performance  of  each  method  was  rated  against  two  criteria,  the  accuracy  and 
stability  of  the  EACs  calculated  with  the  method.  The  MAPE  [Equation  (28)]  was  used 
as  the  measure  of  accuracy,  and  the  SDAPE  [Equation  (29)]  was  used  as  the  measure  of 
stability.  Because  the  performance  of  the  methods  was  rated  against  two  criteria,  it  was 
necessary  to  develop  a  combined  measure  of  performance.  This  was  done  by  separately 
ranking  the  methods  according  to  increasing  values  of  the  MAPE  and  SDAPE  and  then 
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summing  the  individual  rankings  to  arrive  at  the  combined  score  for  each  method.  The 
method  with  the  lowest  score  was  classed  as  the  best  performing  method. 

The  first  step  in  comparing  the  performance  of  the  EAC  methods  was  to  calculate  an 
EAC  for  every  method  using  the  cost  data  from  each  reporting  period  on  all  88  contracts. 
Next,  the  MAPE,  SDAPE,  and  combined  score  were  calculated  for  each  EAC  method 
using  the  complete  set  of  data  to  determine  the  best  performing  method  overall.  Then, 
the  data  were  divided  into  subsets  according  to  four  stages  (quartiles)  of  contract 
completion  (0%  to  25%  complete,  >25%  to  50%  complete,  >50%  to  75%  complete,  and 
>75%  to  100%  complete)  and  the  MAPE  values,  SDAPE  values,  and  combined  scores  for 
each  method  were  calculated  within  each  of  these  data  subsets.  This  was  done  to  test  for 
the  sensitivity  of  the  overall  results  to  stage  of  contract  completion. 

The  overall  results  were  also  tested  for  sensitivity  to  system  type  (Aircraft,  Armored 
Vehicle,  Ballistic  Missile,  Electronic  Warfare,  Helicopter,  Other  Missile,  Satellite,  Ship, 
and  Torpedo),  program  phase  (Development,  Production,  Other),  contract  type  (Cost 
Plus,  Fixed  Price,  Mixed  Type),  and  DoD  service  component  (Army,  Air  Force,  and 
Navy).  To  do  this,  the  data  were  divided  into  the  appropriate  subsets  and  the  MAPE 
values,  SDAPE  values,  and  combined  scores  for  each  EAC  method  were  calculated 
within  each  of  these  data  subsets.  The  results  for  each  of  these  categories,  were  also 
tested  for  sensitivity  to  stage  of  contract  completion.  The  final  stage  of  analysis  was 
testing  for  sensitivity  to  the  effects  of  inflation.  To  do  this,  the  complete  analysis  was 
first  done  using  non-inflation  adjusted  data  (TY$),  and  then  repeated  using  data  which 
was  adjusted  to  CY95S. 
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Overall  the  hypothesis  was  supported  by  the  results.  The  index-based  methods 
performed  significantly  better  than  the  nonlinear  regression-based  methods  at  the  overall 
level,  and  within  each  of  the  data  subsets  used  in  the  sensitivity  analysis.  There  were  only 
four  cases  where  a  nonlinear  regression-based  method  was  found  to  be  the  top  performer. 
The  four  cases  were:  1)  the  Beta  method  using  inflation  adjusted  data  for  the  Helicopter 
contracts  in  the  fourth  quartile;  2)  the  Beta  method  using  inflation  adjusted  data  for  the 
Satellite  contracts  in  the  fourth  quartile;  3)  the  Beta  method  using  both  inflation  adjusted 
and  non-inflation  adjusted  data  for  the  Other  phase  contracts  in  the  fourth  quartile;  and,  4) 
the  Rayleigh  Least  Squares  Linear  Regression  method  using  inflation  adjusted  data  in  the 
third  quartile.  Unfortunately,  in  each  of  these  cases  the  methods  were  poor  performers 
during  the  earlier  stages  of  contract  completion  which  limits  their  usefulness  as 
management  tools. 

Overall,  the  best  performing  EAC  method  was  found  to  be  the  index-based  method 
using  the  Composite  Index  (0.2SPIcum+0.8CPICUm)-  This  result  held  for  both  the  inflation 
adjusted  data  and  the  non-inflation  adjusted  data.  This  method  was  also  a  top  performer 
in  the  first  quartile  for  the  inflation  adjusted  data,  and  the  first,  second,  and  fourth 
quartiles  for  the  non-inflation  adjusted  data.  It  is  important  to  note,  however,  that  the  top 
performing  method  was  sensitive  to  all  of  the  moderator  variables  that  were  investigated 
in  the  sensitivity  analysis  and  no  single  EAC  method  was  always  the  top  performer. 
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Discussion  of  Findings 

The  results  of  this  study  are  significant  in  three  ways.  First,  they  support  the 
continued  use  of  the  index-based  EAC  methods  by  DoD  managers  and  cost  analysts. 
Because  they  are  easy  to  use,  the  index-based  EAC  methods  will  likely  continue  to  be  the 
most  popular  methods  until  newer  methods  are  developed  that  can  demonstrate  better 
performance  levels. 

Second,  the  results  of  this  study  support  the  results  of  previous  studies  with  regards 
to  the  sensitivity  of  results  to  such  factors  as  stage  of  contract  completion,  system  type, 
program  phase,  contract  type,  DoD  service  component,  and  inflation  effects.  Given  the 
sensitivity  of  the  results  to  these  factors,  no  single  EAC  method  should  be  relied  on  in 
every  situation.  While  the  results  of  this  study  can  serve  as  a  guide  for  the  program 
manager  or  cost  analyst  selecting  an  EAC  method  for  their  program,  the  practice  of  using 
multiple  methods  to  calculate  a  range  of  EACs  cannot  be  ignored. 

Third,  although  the  nonlinear  regression-based  EAC  methods  were  not  found  to  be 
more  accurate  than  the  index-based  methods,  the  results  of  this  study  still  provide  insight 
into  the  relative  accuracy  of  the  EAC  methods  that  were  tested.  The  detailed  results 
provided  in  Appendix  H  provide  a  useful  tool  which  can  be  used  by  program  managers 
and  cost  analysts  to  assess  the  potential  accuracy  of  a  particular  EAC  method  given  their 
particular  situation. 
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Recommendati  ons 


This  study  has  demonstrated  the  superiority  of  the  index -based  EAC  methods  over 
several  of  the  newer  nonlinear  regression-based  methods.  While  this  is  significant,  the 
overall  accuracy  of  the  index-based  methods  during  the  early  stages  of  contract 
completion  is  still  relatively  poor.  Research  is  still  needed  to  find  EAC  methods  which 
can  accurately  predict  the  CAC  early  in  a  program’s  life  at  a  point  where  changes  can  be 
made  and  actions  taken  to  correct  problems  and  prevent  costly  overruns. 

Finally,  as  new  EAC  methods  are  developed,  there  will  be  a  continuing  need  for 
comparative  studies  such  as  this  one,  to  evaluate  the  performance  of  the  new  methods 
relative  to  the  popular  index-based  methods.  These,  future  studies  will  be  able  to  take 
advantage  of  updated  versions  of  the  DAES  database  and  will  likely  be  able  to  include  a 
much  larger  sample  of  contracts  than  was  analyzed  in  this  study,  thereby  increasing  the 
significance  of  the  results. 
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Appendix  A:  Cost/Schedule  Control  Systems  Definitions 


This  Appendix  presents  the  official  definitions  of  C/SCS  terms  which  were  extracted 
from  DoDI  5000.2.  Although  some  of  the  terms  presented  here  are  not  used  in  the  body 
of  this  thesis,  many  of  the  terms  are  used  to  define  other  terms.  Therefore,  all  of  the 
terms  from  5000.2  have  been  included  in  this  Appendix. 


1.  Actual  Cost  of  Work  Performed  (ACWP).  The  cost  incurred  and 
recorded  in  accomplishing  the  work  performed  within  a  given  time 
period. 

2.  Actual  Direct  Costs.  Those  costs  identified  specifically  with  a  contract, 
based  upon  the  contractor’s  cost  identification  and  accumulation  system 
as  accepted  by  the  cognizant  Defense  Contract  Audit  Agency 
representatives.  (See  definition  14  below.) 

3.  Allocated  Budget.  (See  definition  32,  below.) 

4.  Applied  Direct  Cost.  The  amount  recognized  in  the  time  period 
associated  with  the  consumption  of  labor,  material,  and  other  direct 
resources,  without  regard  to  the  date  of  commitment  or  the  date  of 
payment.  These  amounts  are  to  be  charged  to  work-in-progress  in  the 
time  period  that  any  one  of  the  following  occurs: 

a.  When  labor,  material,  and  other  direct  resources  are  actually 
consumed. 

b.  When  material  resources  are  withdrawn  from  inventory  for  use. 

c.  When  material  resources  are  received  that  are  identified  uniquely  to 
the  contract  and  scheduled  for  use  within  60  days. 

d.  When  major  components  or  assemblies  are  received  on  a  line  flow 
basis  that  are  identified  specifically  and  uniquely  to  a  single  serially 
numbered  end  item. 

5.  Apportioned  Effort.  Effort  that  is  not  readily  divisible  into  work 
packages,  but  is  related  proportionally  to  measured  effort. 
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6.  Authorized  Work.  Effort  that  has  been  definitized  and  is  on  contract, 
plus  that  for  which  definitized  contract  costs  have  not  been  agreed  to,  but 
for  which  written  authorization  has  been  received. 

7.  Baseline.  (See  definition  24,  below.) 

8.  Budgeted  Cost  of  Work  Performed  (BCWPf  The  sum  of  the  budgets  for 
completed  work  packages  and  completed  portions  of  open  work 
packages,  plus  the  applicable  portion  of  the  budgets  for  level  of  effort 
and  apportioned  effort. 

9.  Budgeted  Cost  of  Work  Scheduled  fBCWS).  The  sum  of  budgets  for  all 
work  packages,  planning  packages,  etc.,  scheduled  to  be  accomplished 
(including  in-process  work  packages),  plus  the  amount  of  level-of-effort 
and  apportioned  effort  scheduled  to  be  accomplished  within  a  given  time 
period. 

10.  Budgets  for  Work  Packages.  (See  definition  36,  below.) 

1 1 .  Contract  Budget  Base.  The  negotiated  contract  cost  plus  the  estimated 
cost  of  authorized  unpriced  work. 

12.  Contractor.  An  entity  in  private  industry  which  enters  into  contracts 
with  the  Government.  In  this  Instruction,  the  word  also  may  apply  to 
Government-owned,  Government-operated  activities  that  perform  work 
on  defense  programs. 

13.  Cost  Account.  A  management  control  point  at  which  actual  costs  may 
be  accumulated  and  compared  to  the  budgeted  cost  of  the  work 
performed.  A  cost  account  is  a  natural  control  point  for  cost/schedule 
planning  and  control,  since  it  represents  the  work  assigned  to  one 
responsible  organizational  element  on  one  contract  work  breakdown 
structure  element. 

14.  Direct  Costs.  Any  costs  that  may  be  identified  specifically  with  a  particular 
final  cost  objective.  This  term  is  explained  in  the  Federal  Acquisition 
Regulation. 

15.  Estimate  at  Comnletion  IE  AC!.  Actual  direct  costs,  plus  indirect  costs 
allocable  to  the  contract,  plus  estimate  of  costs  (direct  and  indirect)  for 
authorized  work  remaining. 
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1 6.  Indirect  Costs.  Costs,  which  because  of  their  incurrence  for  common  or 
joint  objectives,  are  not  subject  readily  to  treatment  as  direct  costs.  This 
term  is  further  defined  in  the  Federal  Acquisition  Regulation. 

17.  Initial  Budget.  (See  definition  22,  below.) 

18.  Internal  Replanning.  Replanning  actions  performed  by  the  contractor  for 
remaining  effort  within  the  recognized  total  allocated  budget. 

1 9.  Level-of-Effort  (LOE).  Effort  of  a  general  or  supportive  nature  that  does 
not  produce  definite  end  products. 

20.  Management  Reserve  or  Management  Reserve  Budget.  An  amount  of 
the  total  allocated  budget  withheld  for  management  control  purposes, 
rather  than  designated  for  the  accomplishment  of  a  specific  task  or  set  of 
tasks.  It  is  not  a  part  of  the  performance  measurement  baseline. 

21 .  Negotiated  Contract  Cost.  The  estimated  cost  negotiated  in  a  cost  plus 
fixed  fee  contract,  or  the  negotiated  contract  target  cost  in  either  a  fixed 
price  incentive  contract  or  a  cost  plus  incentive  fee  contract. 

22.  Original  Budget.  The  budget  established  at,  or  near,  the  time  that  the 
contract  was  signed  and  based  on  the  negotiated  contract  cost. 

23.  Overhead.  (See  definition  1 6,  above.) 

24.  Performance  Measurement  Baseline.  The  time  phased  budget  plan 
against  which  contract  performance  is  measured.  It  is  formed  by  the 
budgets  assigned  to  scheduled  cost  accounts  and  the  applicable  indirect 
budgets.  For  future  effort,  not  planned  to  the  cost  account  level,  the 
performance  measurement  baseline  also  includes  budgets  assigned  to 
higher  level  contract  work  breakdown  structure  elements  and 
undistributed  budgets.  It  equals  the  total  allocated  budget  less 
management  reserve. 

25.  Performing  Organization.  A  defined  unit  within  the  contractor’s 
organizational  structure,  which  applies  the  resources  to  perform  the 
work. 

26.  Planning  Package.  A  logical  aggregation  of  far  term  work  within  a  cost 
account  which  may  be  identified  and  budgeted  in  early  baseline 
planning,  but  is  not  yet  defined  into  work  packages. 
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27.  Procuring  Activity.  The  subordinate  command  in  which  the 
Procurement  Contracting  Officer  is  located.  It  may  include  the  program 
office,  related  functional  support  offices,  and  procurement  offices. 
Examples  of  procuring  activities  are  the  Army  Missile  Command,  Naval 
Sea  Systems  Command,  and  Air  Force  Electronic  Systems  Division. 

28.  Replanning.  (See  definition  1 8,  above.) 

29.  Reprogramming.  Replanning  of  the  effort  remaining  in  the  contract, 
resulting  in  a  new  budget  allocation  that  exceeds  the  contract  budget 
base. 

30.  Responsible  Organization.  A  defined  unit  within  the  contractor’s 
organizational  structure  that  is  assigned  responsibility  for  accomplishing 
specific  tasks. 

31.  Significant  Variances.  Those  differences  between  planned  and  actual 
performance  requiring  further  review,  analysis,  or  action.  Thresholds 
should  be  established  as  to  the  magnitude  of  variances  that  will  require 
variance  analysis,  and  the  thresholds  should  be  revised  as  needed  to 
provide  meaningful  analysis  during  execution  of  the  contract. 

32.  Total  Allocated  Budget.  The  sum  of  all  budgets  allocated  to  the 
contract.  Total  allocated  budget  consists  of  the  performance 
measurement  baseline  and  all  management  reserve.  The  total  allocated 
budget  will  reconcile  directly  to  the  contract  budget  base.  Any 
differences  will  be  documented  as  to  quantity  and  cause. 

33 .  Undistributed  Budget.  Budget  applicable  to  contract  effort  that  has  not 
yet  been  identified  to  contract  work  breakdown  structure  elements  at,  or 
below,  the  lowest  level  of  reporting  to  the  Government. 

34.  Variances.  (See  definition  31,  above.) 

35.  Work  Breakdown  Structure  (WBS).  (See  Section  6-B.) 

36.  Work  Package  Budgets.  Resources  that  are  assigned  formally  by  the 
contractor  to  accomplish  a  work  package,  expressed  in  dollars,  hours, 
standards,  or  other  definitive  units. 

37.  Work  Packages.  Detailed  tasks  or  material  items  identified  by  the 
contractor  for  accomplishing  work  required  to  complete  the  contract.  A 
work  package  has  the  following  characteristics: 
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a.  It  represents  unit  of  work  at  levels  where  work  is  performed. 

b.  It  is  clearly  distinguishable  from  all  other  work  packages. 

c.  It  is  assignable  to  a  single  organizational  element. 

d.  It  has  scheduled  start  and  completion  dates  and,  as  applicable, 
interim  milestones;  all  of  which  are  representative  of  physical 
accomplishment. 

e.  It  has  a  budget  or  assigned  value  expressed  in  terms  of  dollars, 
manhours,  or  other  measurable  units. 

f.  Its  duration  is  limited  to  a  relatively  short  time  span  or  it  is 
subdivided  by  discrete  value  milestones  to  ease  the  objective 
measurement  of  work  performed. 

g.  It  is  integrated  with  detailed  engineering,  manufacturing,  or  other 
schedules.  (Department  of  Defense,  1991:1  l-B-2-1  -  ll-B-2-4) 
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Appendix  B:  List  of  Contracts  I  Jsed  in  Analysis 


Program  Name:  A- 10  (THUNDERBOLT  II) 

PNO:  102 

System  Type:  Aircraft 

DoD  Service  Component:  Air  Force 

PD:  Airframe  (Options  3-6) 

CNO:  1 

Program  Phase:  Other 
Contract  Type:  FPI 

Program  Name:  EF-1 1 1 A  (TJS) 

PNO:  104 

System  Type:  Electronic  Warfare 
DoD  Service  Component:  Air  Force 

PD:  Tactical  Jamming  System 
CNO:  1 

Program  Phase:  Development 
Contract  Type:  CP 

PD:  Production 
CNO:  2 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  PATRIOT  Surface-to-Air  Missile  System 
PNO:  106 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  FY85  Missile  Prod 
CNO:  2 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Init  Prod  Fac  (Buy  7) 

CNO:  3 

Program  Phase:  Production 
Contract  Type:  CPIF 
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PD:  FY86  Missile  Prod 
CNO:  4 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  FY83  Eng  Services 
CNO:  92 

Program  Phase:  Other 
Contract  Type:  CP 

PD:  FY81  Missile  Prod 
CNO:  94 

Program  Phase:  Production 
Contract  Type:  CP 

Program  Name:  PERSHING  II  Field  Artillery  Missile  System 
PNO:  107 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  MSL  &  GSE  Development 
CNO:  1 

Program  Phase:  Development 
Contract  Type:  CP 

PD:  MSL  &  GSE  Production 
CNO:  2 

Program  Phase:  Production 
Contract  Type:  CP 

Program  Name:  ALCM  (Air-Launched  Cruise  Missile)  (AGM-86B) 

PNO:  111 

System  Type:  Other  Missile 
DoD  Service  Component:  Navy 

PD:  FY81  Prod  Air  Vehicle 
CNO:  54 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  CH-47D  (CHINOOK  Helicopter  Modernization  Program) 
PNO:  115 

System  Type:  Helicopter 
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DoD  Service  Component:  Army 

PD:  FY82  Medium  Lift  Heli 
CNO:  95 

Program  Phase:  Other 
Contract  Type:  CP 

PD:  FY81  Medium  Lift  Heli 
CNO:  96 

Program  Phase:  Other 
Contract  Type:  FPI 

PD:  IPF/LLT 
CNO:  97 

Program  Phase:  Other 
Contract  Type:  FPI 

PD:  PEP 
CNO:  98 

Program  Phase:  Other 
Contract  Type:  CP 

Program  Name:  C/MH-53E  Super  Stallion  Helicopter 
PNO:  116 

System  Type:  Helicopter 
DoD  Service  Component:  Navy 

PD:  FY78  Buy  6  A/C 
CNO:  98 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  DSCS  III  (Defense  Satellite  Communication  System)  SHF  Space 

Segment 

PNO:  120 

System  Type:  Satellite 

DoD  Service  Component:  Air  Force 

PD:  IABS  Development 
CNO:  2 

Program  Phase:  Development 
Contract  Type:  FPIF 

Program  Name:  F-14D  TOMCAT  All-Weather,  Carrier-Based  Air-Superiority  Fighter 
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PNO:  123 

System  Type:  Aircraft 

DoD  Service  Component:  Navy 


PD:  Airframe  Production 
CNO:  96 

Program  Phase:  Production 
Contract  Type:  FFP 

Program  Name:  F-15  EAGLE  Air-Superiority/Dual-Role  Fighter 
PNO:  124 

DoD  Service  Component:  Air  Force 

PD:  ALQ-135  Update  Lot  III 
CNO:  8 

System  Type:  Electronic  Warfare 
Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Engine  (Lot  VIII) 

CNO:  91 

System  Type:  Aircraft 
Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Airframe  Development 
CNO:  93 

System  Type:  Aircraft 
Program  Phase:  Development 
Contract  Type:  CP 

PD:  FY77  Prod  &  Support 
CNO:  95 

System  Type:  Aircraft 
Program  Phase:  Production 
Contract  Type:  FPIF 

Program  Name:  F-16  FIGHTING  FALCON  Multi-Mission  Fighter  (Includes  F-16 
Derivatives) 

PNO:  125 

PD:  ASPJ  Lot  I  Production 
CNO:  24 

System  Type:  Electronic  Warfare 
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Program  Phase:  Production 

Contract  Type:  FPIF 

DoD  Service  Component:  Navy 

PD:  Engine  (Lot  VIII) 

CNO:  88 

System  Type:  Aircraft 

Program  Phase:  Production 

Contract  Type:  FPI 

DoD  Service  Component:  Air  Force 

PD:  FY80-  175  A/C 
CNO:  90 

System  Type:  Aircraft 

Program  Phase:  Production 

Contract  Type:  FPI 

DoD  Service  Component:  Air  Force 

PD:  Airframe  Development 
CNO:  96 

System  Type:  Aircraft 

Program  Phase:  Development 

Contract  Type:  FPI 

DoD  Service  Component:  Air  Force 

Program  Name:  FFG-7  PERRY  Class  Frigate 

PNO:  127 

System  Type:  Ship 

DoD  Service  Component:  Navy 

PD:  FY79  Buy  (3  Ships) 

CNO:  81 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Three  Ships 
CNO:  91 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  BRADLEY  FVS  (Fighting  Vehicle  System  -  M2/M3) 
PNO:  128 

System  Type:  Armored  Vehicle 
DoD  Service  Component:  Army 
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PD:  25MM  Gun  Option  1 
CNO:  88 

Program  Phase:  Other 
Contract  Type:  FPI 

Program  Name:  MLRS  (Multiple-Launch  Rocket  System) 

PNO:  130 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  FY81  LRP  Buy  II A/V 
CNO:  91 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Maturation  R&D 
CNO:  97 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  SH-60B  SEAHAWK  LAMPS  MK  III  (Light  Airborne  Multi-Purpose 
System) 

PNO:  134 

System  Type:  Helicopter 
DoD  Service  Component:  Navy 

PD:  Engine  Development 
CNO:  94 

Program  Phase:  Development 
Contract  Type:  CP 

PD:  Airframe  Development 
CNO:  95 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  MAVERICK  Close  Air  Support  Weapon  System  (HR)  (AGM-65D/F/G) 
PNO:  137 

System  Type:  Other  Missile 
DoD  Service  Component:  Air  Force 

PD:  Raytheon  Production 
CNO:  1 
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Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Missile  Development 
CNO:  99 

Program  Phase:  Development 
Contract  Type:  FPIF 

Program  Name:  MK-48  Advanced  Capability  (ADCAP)  Torpedo 
PNO:  138 

System  Type:  Torpedo 
DoD  Service  Component:  Navy 

PD:  L3  Test  Equipment 
CNO:  5 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  SIDEWINDER  AIM-9L,  NV 
PNO:  144 

System  Type:  Other  Missile 
DoD  Service  Component:  Navy 

PD:  Guidance  Development 
CNO:  97 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  STINGER  Man-Portable  Surface-to-Air  Missile  (FIM-92A/B) 
PNO:  150 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  STINGER  RMP  RDT&E 
CNO:  92 

Program  Phase:  Development 
Contract  Type:  FPI 

PD:  FY82  Prod 
CNO:  94 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  UH-60A  BLACKHAWK  Helicopter 
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PNO:  156 

System  Type:  Helicopter 
DoD  Service  Component:  Army 


PD:  4th  Year  Production 
CNO:  85 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Third  YR  Airframe  Prod 
CNO:  86 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Second  YR  Airframe  Prod 
CNO:  87 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  FSD-Maturity 
CNO:  90 

Program  Phase:  Development 
Contract  Type:  CP 

PD:  Engine  Production 
CNO:  94 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  First  Year  Airframe  Prod 
CNO:  95 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  Engine  Development 
CNO:  98 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  Ml/Ml  A1  ABRAMS  Tank  (w/ 120mm  Gun) 
PNO:  157 

System  Type:  Armored  Vehicle 
DoD  Service  Component:  Army 
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PD:  Tank  Development 
CNO:  87 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  AH-64A  APACHE  (Advanced  Attack  Helicopter) 

PNO:  158 

System  Type:  Helicopter 
DoD  Service  Component:  Army 

PD:  TADS/PNVS  Lot  III 
CNO:  86 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Engine  Development 
CNO:  98 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  CG-47  (TICONDEROGA  Class  AEGIS  Guided  Missile  Cruiser) 

PNO:  159 

System  Type:  Ship 

DoD  Service  Component:  Navy 

PD:  CG  66/8  Construction 
CNO:  5 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  CG  67  Construction 
CNO:  6 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  NAVSTAR  GPS/UE  (Global  Positioning  System/User  Equipment) 
PNO:  166 

System  Type:  Satellite 

DoD  Service  Component:  Air  Force 

PD:  User  Equipment 
CNO:  95 

Program  Phase:  Other 
Contract  Type:  FPI 
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Program  Name:  B-1B  Strategic  Bomber 
PNO:  168 

DoD  Service  Component:  Air  Force: 

PD:  Offensive  Avionics  Lot  2 
CNO:  2 

System  Type:  Electronic  Warfare 
Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Offensive  Avionics  Lot  5 
CNO:  3 

System  Type:  Electronic  Warfare 
Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Defensive  Avionics  Lot  2 
CNO:  93 

System  Type:  Electronic  Warfare 
Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Offensive  Avionics  Lot  1 
CNO:  99 

System  Type:  Electronic  Warfare 
Program  Phase:  Production 
Contract  Type:  FPIF 

Program  Name:  TRIDENT  II  D5  SLBM  (Sea  Launched  Ballistic  Missile)  (UGM-133A) 
PNO:  178 

System  Type:  Ballistic  Missile 
DoD  Service  Component:  Navy 

PD:  Guidance  System  Dev 
CNO:  4 

Program  Phase:  Development 
Contract  Type:  CPFF 

PD:  Launcher  Oper  System 
CNO:  11 

Program  Phase:  Other 
Contract  Type:  CPIF/FF 

Program  Name:  MK-50  Advanced  Lightweight  Torpedo  (ALWT) 
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PNO:  192 

System  Type:  Torpedo 
DoD  Service  Component:  Navy 

PD:  MK50FSED 
CNO:  2 

Program  Phase:  Development 
Contract  Type:  CPAF/FF 

PD:  MK  50  Torpedo  LRIP  I 
CNO:  3 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  MK  50  Torpedo  LRIP  II 
CNO:  5 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  ATACMS  (Army  Tactical  Missile  System) 

PNO:  211 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  FSD  LCHR  &  GSE  Integ 
CNO:  2 

Program  Phase:  Development 
Contract  Type:  FPIF 

Program  Name:  V-22  OSPREY  Joint  Advanced  Vertical  Lift  Aircraft 
PNO:  212 

System  Type:  Aircraft 

DoD  Service  Component:  Navy 

PD:  Prelim  Design  Stage  I 
CNO:  99 

Program  Phase:  Development 
Contract  Type:  CP 

Program  Name:  LCAC  (Landing  Craft,  Air  Cushion) 

PNO:  216 

System  Type:  Ship 

DoD  Service  Component:  Navy 
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PD:  LCAC  24-33  Construction 
CNO:  3 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  LCAC  15-23  Construction 
CNO:  4 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  LCAC  37-48  Construction 
CNO:  5 

Program  Phase:  Production 
Contract  Type:  FPI 

PD:  LCAC  34-36  Construction 
CNO:  6 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  PEACEKEEPER  ICBM  (Intercontinental  Ballistic  Missile)  (MX;  LGM- 
1 18A) 

PNO:  225 

System  Type:  Ballistic  Missile 
DoD  Service  Component:  Air  Force 

PD:  Assembly  &  Checkout  A&C 
CNO:  10 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Inertial  Meas  Unit  FY84 
CNO:  12 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Stage  II,  FY85 
CNO:  16 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Stage  IV,  FY85 
CNO:  19 

Program  Phase:  Production 
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Contract  Type:  FPIF 


PD:  Stage  III,  FY85 
CNO:  22 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Guidance  &  Control,  FY85 
CNO:  25 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Reentry  System,  FY88/89 
CNO:  31 

Program  Phase:  Production 
Contract  Type:  Mixed  (FPIF/CPF) 

PD:  Reentry  System  &  Reentry  Vehicle,  FY84 
CNO:  59 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Stage  III 
CNO:  68 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Stage  I 
CNO:  70 

Program  Phase:  Production 
Contract  Type:  FPIF 

PD:  Reentry  Sys  Follow-on 
CNO:  72 

Program  Phase:  Development 
Contract  Type:  FPIF 

PD:  TGG  (Follow-on  Dev) 

CNO:  83 

Program  Phase:  Development 
Contract  Type:  FPIF 

PD:  SFIR  (Follow-on  Dev) 

CNO:  84 
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Program  Phase:  Development 
Contract  Type:  FPI 

PD:  MK-12A 
CNO:  94 

Program  Phase:  Other 
Contract  Type:  FPI 

Program  Name:  OTH-B  (Over-the-Horizon  Backscatter  Radar) 

PNO:  227 

System  Type:  Electronic  Warfare 
DoD  Service  Component:  Air  Force 

PD:  AN/FPS-1 1 8  Radar  Sec  4 
CNO:  2 

Program  Phase:  Production 
Contract  Type:  FPIF 

Program  Name:  SMALL  MISSILE  (ICBM) 

PNO:  267 

System  Type:  Ballistic  Missile 
DoD  Sendee  Component:  Air  Force 

PD:  G&C  Integration  (FSD) 

CNO:  3 

Program  Phase:  Development 
Contract  Type:  FPIF/AF 

PD:  Hard  Mobile  Basing 
CNO:  11 

Program  Phase:  Development 
Contract  Type:  FPIF/AF 

Program  Name:  LANCE  (MGM-52C) 

PNO:  304 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 

PD:  Fourth  Buy 
CNO:  99 

Program  Phase:  Production 
Contract  Type:  FPI 

Program  Name:  STINGER-Reprogrammable  Microprocessor  (RMP) 
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PNO:  532 

System  Type:  Other  Missile 
DoD  Service  Component:  Army 


PD:  RMP  FY8 5 -Production 
CNO:  2 

Program  Phase:  Production 
Contract  Type:  FPI/FFP 

PD:  RMP  FY86-Production 
CNO:  3 

Program  Phase:  Production 
Contract  Type:  FPI 


*********************************************************************** 

UDF  Name:  IndexEAC 
Written  by:  Capt  Todd  D.  Nystrom 
Date:  5  May  1995 

This  Microsoft®  Excel  user  defined  function  (UDF)  calculates  index-based  EACs 
for  eleven  different  performance  indices 

Input  Variables: 

row  first:  indicates  the  first  row  of  data  for  this  contract 
row_current:  indicates  the  row  number  of  cell  calling  this  function 
index_flag:  indicates  which  index  to  use  in  calculating  the  EAC 

1  =  CPI3  where  CPI3  =  BCWP3/ACWP3 

2  =  CPI6  where  CPI6  =  BCWP6/ACWP6 

3  =  CPIcum  where  CPIcum  =  BCWPcum/ACWPcum 

4  =  SPI3  where  SPI3  =  BCWP3/BCWS3 

5  =  SPI6  where  SPI6  =  BCWP6/BCWS6 

6  =  SPIcum  where  SPIcum  =  BCWPcum/BCWScum 

7  =  SCI3  where  SCI3  =  SPI3*CPI3 

8  =  SCI6  where  SC16  =  SPI6*CPI6 

9  =  SCIcum  where  SCIcum  =  SPIcum*CPIcum 

1 0  =  SCIspecial  where  SCIspecial  =  SPIcum  x  CPI6 

1 1  =  Composite  where  Composite  =  .8CPIcum  +  .2  SPIcum 

Note:  contract  startdate,  BAC,  report_date,  BCWScum,  BCWPcum,  and 
ACWPcum  are  also  inputs  to  this  function,  however,  these  are  accessed  directly  from 
the  spreadsheet  rather  that  being  provided  as  specific  inputs  to  the  function.  The 
location  of  these  values  within  the  spreadsheet  is  critical.  The  column  where  each 
input  must  be  located  is  follows: 

start  date:  column  10  (J) 

BAC:  column  14  (N) 

report_date:  column  4  (D) 

BCWScum:  column  11  (K) 

BCWP  cum:  column  12  (L) 

ACWP  cum:  column  13  (M) 

Output:  this  function  outputs  the  index-based  EAC  for  the  index  indicated  by  index_flag 
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to  the  cell  calling  the  function,  and  EAC  is  calculated  using  the  following: 

EAC  =  ACWPcum  +  ((BAC  -  BCWPcum)  /  Index) 

* *********%%********************************************************%** 


Function  Index_EAC(row_f!rst,  row_current,  index_flag) 

'declare  date  variables 
Dim  startdate  As  Date 

'dimension  arrays 
Dim  report_date(60)  As  Date 
Dim  BCWS_cum(60)  As  Double 
Dim  BCWP_cum(60)  As  Double 
Dim  ACWP_cum(60)  As  Double 

'initialize  variables 
flag_3_mo  =  1 
flag_6_mo  =  1 

'determine  number  of  data  points  (n)  for  this  contract  at  current  time 
n  -  row_current  -  row_first  +  1 

'contract  start  date  must  be  in  column  1 0  (J)  of  spreadsheet 
startdate  =  cells(row_current,  10) 

'BAC  values  must  be  in  column  1 4  (N)  of  spreadsheet 
BAC  =  cells(row_current,  14) 

'place  report  dates,  BCWScum,  BCWP  cum,  and  ACWP  cum  values  in  arrays 
For  i  =  1  To  n 

'report  dates  must  be  in  column  4  (D)  of  spreadsheet 
report_date(i)  =  cells(row_first  +  i  -  1,4) 

'cumulative  BCWS  values  must  be  in  column  1 1  (K)  of  spreadsheet 
BCWS  cum(i)  =  cells(row_first  +  i  -  1, 1 1) 

'cumulative  BCWP  values  must  be  in  column  12  (L)  of  spreadsheet 
BCWP  cum(i)  =  cells(row_first  +  i  -  1, 12) 

'cumulative  AC WP  values  must  be  in  column  1 3  (M)  of  spreadsheet 
ACWP_cum(i)  =  cells(row_first  +  i  - 1, 13) 

Next  i 

'calculate  3  or  6  month  BCWS,  BCWP,  ACWP  values  as  required  for  Index 
'indicated  by  indexflag 

'  1  (CPI3),  4  (SPI3),  and  7  (SCI3)  require  3  month  data 


106 


'  2  (CPI6),  5  (SPI6),  8  (SCI6),  and  10  (SPIcum  x  CPI6)  require  6  month  data 
'  Remaining  indices  3  (CPIcum),  6  (SPIcum),  9  (SCIcum),  and  1 1  (Composite) 

'  require  only  cumulative  data. 

Select  Case  index  flag 

Case  1 , 4,  7  'indices  requiring  3  month  data 

'calculates  3  month  non-cumulative  BCWS,  BCWP,  and  ACWP  values 
intflag  =  1  'default  interpolate  flag  to  1  (where  l=interpolate) 

For  i  =  1  To  n 

'check  for  actual  report  date  within  plus  or  minus  1 5  days  of 
'current  report  date  -  3  months 

If  report  date(i)  -  15  <=  report  date(n)  -  d_3(report_date(n))  And  report_date(i) 
+  1 5  >=  report  date(n)  -  d_3(report_date(n))  Then 

int_flag  =  0  'don't  need  to  interpolate  since  actual  exists 
BCWS3  =  BCWS_cum(n)  -  BCWS_cum(i) 

BCWP  3  =  BCWP  cum(n)  -  BCWP_cum(i) 

ACWP  3  =  ACWP  cum(n)  -  ACWP_cum(i) 

End  If 
Next  i 

'if  int _flag=l  then  values  must  be  interpolated 
If  int  flag  =  1  Then 

'if  3  month  prior  date  is  earlier  than  startdate  then  3 
'month  EACs  cannot  be  calculated 
If  report  date(n)  -  d_3(report_date(n))  <  start_date  Then 
flag_3_mo  =  0 
End  If 

'if  3  month  prior  date  is  earlier  than  first  report  then  use  first  report  and  zero 
'values  in  the  interpolation 

If  reportdate(n)  -  d_3(report_date(n))  <  report_date(l)  Then 

BCWS  3  =  BCWS_cum(n)  -  interpolate(BCWS_cum(l),  0,  report_date(l), 
report  date(n)  -  d_3(report_date(n)),  start_date) 

BCWP  3  =  BCWP  cum(n)  -  interpolate(BC WP_cum(  1 ),  0,  report_date(  1 ), 
report  date(n)  -  d_3(report_date(n)),  start  date) 

AC WP_3  =  AC WP_cum(n)  -  interpolate(AC WP_cum(  1 ),  0,  report_date(  1 ), 
report  date(n)  -  d_3(report_date(n)),  start  date) 

'otherwise  determine  actuals  to  use  in  the  interpolation 
Else 

For  i  =  n  To  2  Step  -1 
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If  report_date(i  -  1)  <  report_date(n)  -  d_3(report_date(n))  And 
report  date(i)  >  reportdate(n)  -  d_3(report_date(n))  Then 

BCWS  3  =  BCWS_cum(n)  -  interpolate(BCWS_cum(i),  BCWS_cum(i 

-  1),  reportdate(i),  reportdate(n)  -  d_3(report_date(n)),  report_date(i  - 1)) 

BCWP_3  =  BCWP_cum(n)  -  interpolate(BCWP_cum(i),  BCWP_cum(i 
- 1),  report_date(i),  reportdate(n)  -  d_3(report_date(n)),  report_date(i  - 1)) 

ACWP  3  =  ACWPcum(n)  -  interpolate(ACWP_cum(i),  ACWP_cum(i 

-  1),  reportdate(i),  reportdate(n)  -  d_3(report_date(n)),  report_date(i  -  1)) 

Exit  For 
End  If 
Next  i 
End  If 
End  If 

Case  2,  5,  8, 10  'indices  requiring  6  month  data 

'calculates  6  month  non-cumulative  BCWS,  BCWP,  and  ACWP  values 
int  flag  =  1  'default  interpolate  flag  to  1  (where  ^interpolate) 

For  i  =  1  To  n 

'check  for  actual  report  date  within  plus  or  minus  1 5  days  of  the  current 
'report  date  -  6  months 

If  report  date(i)  - 15  <=  reportdate(n)  -  d_6(report_date(n))  And  report  date(i) 
+  1 5  >=  report  date(n)  -  d_6(report_date(n))  Then 

int  flag  =  0  'don't  need  to  interpolate  since  actual  exists 
BCWS_6  =  BCWS  cum(n)  -  BCWS_cum(i) 

BCWP  6  =  BCWP_cum(n)  -  BCWP_cum(i) 

ACWP  6  =  ACWP_cum(n)  -  ACWP_cum(i) 

End  If 
Next  i 

'if  int_flag  =  1  then  values  must  be  interpolated 
If  int  flag  =  1  Then 

'if  6  month  prior  date  is  earlier  than  start  date  then  6  month  EACs  cannot  be 
'calculated 

If  report  date(n)  -  d_6(report_date(n))  <  start  date  Then 
flag_6_mo  =  0 
End  If 

'if  6  month  prior  date  is  earlier  than  first  report  then  use  first  report  and  zero 
'values  in  the  interpolation 

If  report  date(n)  -  d_6(report_date(n))  <  report  date(l)  Then 
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BCWS  6  =  BCWScum(n)  -  interpolate(BCWS_cum(l),  0,  report  date(l), 
report  date(n)  -  d_6(report_date(n)),  startdate) 

BCWP  6  =  BCWP  cum(n)  -  interpoiate(BCWP_cum(l),  0,  reportdate(l), 
reportdate(n)  -  d_6(report_date(n)),  start_date) 

ACWP_6  =  ACWP_cum(n)  -  interpolate(ACWP_cum(l),  0,  report_date(l), 
report_date(n)  -  d_6(report_date(n)),  start_date) 

'otherwise  determine  actuals  to  use  in  the  interpolation 
Else 

For  i  =  n  To  2  Step  -1 

If  report_date(i  - 1)  <  report_date(n)  -  d_6(report_date(n))  And 
report  date(i)  >  report  date(n)  -  d_6(report_date(n))  Then 

BCWS  6  =  BCWScum(n)  -  interpolate(BCWS_cum(i),  BCWS_cum(i 
-  1),  report  date(i),  report  date(n)  -  d_6(report_date(n)),  report_date(i  -  1)) 

BCWP  6  =  BCWPcum(n)  -  interpolate(BCWP_cum(i),  BCWP_cum(i 
- 1),  report_date(i),  report_date(n)  -  d_6(report_date(n)),  report_date(i  - 1)) 

ACWP  6  =  ACWP_cum(n)  -  interpolate(ACWP_cum(i),  ACWP_cum(i 
- 1),  report  date(i),  report  date(n)  -  d_6(report_date(n)),  report_date(i  -  1)) 

Exit  For 
End  If 
Next  i 
End  If 
End  If 
End  Select 


calculates  EAC  for  Index  indicated  by  index_flag 
1  1  =  CPI3  7  =  SCI3 

1  2  =  CPI6  8  =  SCI6 

'  3  =  CPIcum  9  =  SCIcum 


4  =  SPI3  1 0  =  SPIcum  x  CPI6 

5  =  SPI6  1 1  =  Composite  (.8CPIcum  +  .2SPIcum) 


Select  Case  index  flag 
Case  1  ’CPI3 

'if  any  of  the  listed  conditions  are  met  CPI3  cannot  be  calculated 
If  flag_3_mo  =  0  Or  BCWP  3  <=  0  Or  ACWP_3  <=  0  Then 


IndexEAC  = 

'otherwise  calculate  the  index  and  the  EAC 
Else 

CPI3  =  BCWP  3  /  ACWP_3 

Index_EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  CPI3) 


End  If 


Case  2  'CPI6 

'if  any  of  the  listed  conditions  are  met  CPI6  cannot  be  calculated 
If  flag_6_mo  =  0  Or  BCWP_6  <=  0  Or  ACWP_6  <=  0  Then 
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Index_EAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

CPI6  =  BCWP  6  /  ACWP  6 

Index_EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  CPI6) 
End  If 

Case  3  'CPIcum 

'if  any  of  the  listed  conditions  are  met  CPIcum  cannot  be  calculated 
If  BCWP_cum(n)  <=  0  Or  ACWP_cum(n)  <=  0  Then 
IndexEAC  =  "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

CPIcum  =  BCWP  cum(n)  /  ACWP_cum(n) 

Index  EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  CPIcum) 
End  If 

Case  4  'SPI3 

'if  any  of  the  listed  conditions  are  met  SPI3  cannot  be  calculated 
If  flag_3_mo  =  0  Or  BCWP_3  <=  0  Or  BCWS_3  <=  0  Then 
IndexEAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SPI3  =  BCWP  3  /  BCWS_3 

Index_EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SPI3) 
End  If 

Case  5  'SPI6 

'if  any  of  the  listed  conditions  are  met  SPI6  cannot  be  calculated 
If  flag_6_mo  =  0  Or  BC WP_6  <=  0  Or  BCWS_6  <=  0  Then 
Index_EAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SPI6  =  BCWP  6  /  BCWS  6 

Index  EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SPI6) 
End  If 

Case  6  'SPIcum 

'if  any  of  the  listed  conditions  are  met  SPIcum  cannot  be  calculated 
If  BCWP  cum(n)  <=  0  Or  BCWS_cum(n)  <=  0  Then 
IndexEAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SPIcum  =  BCWPcum(n)  /  BCWS_cum(n) 

Index  EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP  cum(n),  SPIcum) 
End  If 

Case  7  'SCI3  =  SPI3  *  CPI3 

'if  any  of  the  listed  conditions  are  met  SCI3  cannot  be  calculated 
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If  flag_3_mo  =  0  Or  BCWP_3  <=  0  Or  BCWS_3  <=  0  Or  ACWP_3  <=  0  Then 
IndexEAC  =  "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SCI3  =  (BCWP_3  /  BCWS_3)  *  (BCWP_3  /  ACWP_3) 

Index_EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SCI3) 

End  If 

Case  8  'SCI6  =  SPI6  *  CPI6 

'if  any  of  the  listed  conditions  are  met  SCI6  cannot  be  calculated 
If  flag_6_mo  =  0  Or  BCWP_6  <=  0  Or  BCWS_6  <=  0  Or  ACWP_6  <=  0  Then 
Index_EAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SCI6  -  (BCWP  6  /  BCWS  6)  *  (BCWP_6  /  ACWP_6) 

Index  EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SCI6) 

End  If 

Case  9  'SCIcum  =  SPIcum  *  CPIcum 

'if  any  of  the  listed  conditions  are  met  SCIcum  cannot  be  calculated 
If  BCWP_cum(n)  <=  0  Or  BCWS_cum(n)  <=  0  Or  ACWP_cum(n)  <=  0  Then 
IndexEAC  = "" 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SCIcum  =  (BCWPcum(n)  /  BCWS_cum(n))  *  (BCWP_cum(n)  / 
ACWPcum(n)) 

Index  EAC  -  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SCIcum) 

End  If 

Case  1 0  'SCIspecial  =  SPIcum  *  CPI6 

'if  any  of  the  listed  conditions  are  met  SCIspecial  cannot  be  calculated 
If  flag_6_mo  =  0  Or  BCWP_6  <=  0  Or  ACWP_6  <=  0  Or  BCWP_cum(n)  <=  0  Or 
BCWS  cum(n)  <=  0  Then 
IndexEAC  = 

'otherwise  calculate  the  index  and  the  EAC 
Else 

SCIspecial  =  (BCWP_cum(n)  /  BCWS_cum(n))  *  (BCWP_6  /  ACWP_6) 

Index  EAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  SCIspecial) 

End  If 

Case  1 1  'Composite  =  .8CPIcum  +  .2SPIcum 

'if  any  of  the  listed  conditions  are  met  Composite  cannot  be  calculated 
If  BCWP_cum(n)  <=  0  Or  BCWS_cum(n)  <=  0  Or  ACWP_cum(n)  <=  0  Then 
IndexEAC  = 

'otherwise  calculate  the  index  and  the  EAC 
Else 

Composite  =  0.8  *  (BCWP  cum(n)  /  ACWP  cum(n))  +  0.2  *  (BCWP_cum(n)  / 
BCWS_cum(n)) 
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IndexEAC  =  EAC(ACWP_cum(n),  BAC,  BCWP_cum(n),  Composite) 
End  If 
End  Select 

End  Function 


************************************************************************ 
'function  calculates  EAC  using  standard  index-based  EAC  formula  [Equation  (1)] 

4*********************************************************************** 


Function  EAC(ACWPcum,  BACnow,  BCWPcum,  Index) 

EAC  =  ACWPcum  +  ((BACnow  -  BCWPcum)  /  Index) 

End  Function 


4*j|cj(cjic*j|c)|cj|cjtca|cj|ej|c*********************************************************** 

'function  interpolates  BCWS,  BCWP,  or  ACWP  value  based  on  given  input  values 

t*********************************************************************** 

Function  interpolate(value_plus,  valueminus,  date_plus,  date_x,  date  minus) 

interpolate  =  (((value_plus  -  value  minus)  *  (date  x  -  date_minus))  /  (date_plus  - 
date_minus))  +  value_minus 

End  Function 


t*********************************************************************** 

'function  determines  offset  value  in  days  for  3  months  prior 

'Note:  these  offsets  do  not  factor  in  leap  year 

**************  **************************************************** 


Function  d_3(c_date) 

Select  Case  Month(c_date) 

'May 
Case  5 
d_3  =  89 
'March,  April 
Case  3,  4 
d_3  =  90 
'July,  December 
Case  7,  12 
d_3  =  91 

'January,  February,  June,  August,  September,  October,  November 
Case  1, 2,  6,  8,  9,  10,  11 
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d_3  =  92 
End  Select 

End  Function 


4*********************************************************************** 

'function  determines  offset  value  in  days  for  6  months  prior 
Tslote:  these  offsets  do  not  factor  in  leap  year 

4*********************************************************************** 


Function  d_6(c_date) 

Select  Case  Month(c  date) 

'March,  May,  July,  August 
Case  3,  5,  7,  8 
d_6=  181 
'April,  June 
Case  4,  6 
d_6=  182 
'October,  December 
Case  10, 12 
d_6=  183 

'January,  February,  September,  November 
Case  1 ,  2,  9, 1 1 
d_6  = 1 84 
End  Select 

End  Function 
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Appendix  D:  Routine  Used  to  Calculate  Rayleigh  Three  Group  Linear  Regression  EACs 


i*********************************************************************** 

'UDF  Name:  Rayleigh_TG 

'Written  by:  Capt  Todd  D.  Nystrom 

'Date:  1  May  1995 

» 

'This  Microsoft®  Excel  user  defined  function  (UDF)  calculates  an  EAC  by  fitting  actual 
'contract  data  to  a  non-cumulative  Rayleigh  distribution  curve  using  three  group  linear 
'regression 

l 

'Input  Variables: 

'  row_first:  indicates  the  first  row  of  data  for  the  contract 

'  row_current:  indicates  the  row  number  of  cell  calling  this  function 

'  column:  indicates  the  column  number  of  cell  calling  this  function 

i 

'  Note:  x(i)  (tA2)  values  must  be  located  one  column  to  the  left  of  cell 
’  calling  this  function,  and  y(i)  values  (ln[(dACWP/dt)/t])  must  be  located 
'  two  columns  to  the  left  of  cell  calling  this  function. 


'Output:  this  function  outputs  the  three  group  linear  regression  Rayleigh  curve  EAC 
’  to  the  cell  calling  the  function 

i 

t* ********************************************************************** 


Function  Rayleigh_TG(row_first,  row_current,  column) 

'dimension  arrays 

Dim  x(60)  As  Double 

Dim  y(60)  As  Double 

Dim  y_sort(60)  As  Double 

Dim  a(50)  As  Double 

Dim  b(50)  As  Double 

Dim  residual(50,  60)  As  Double 

'determine  number  of  data  points  (n)  for  this  contract 
n  =  row  current  -  row_first  +  1 

'place  t-squared  values  in  x(i),  ln[(dACWP/dt)/t]  values  in  y(i),  and 
'y(i)  values  in  y_sort(i) 
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'there  is  no  need  for  xordered(i)  because  t-squared  (x)  values  are  already  in  ascending 
'order 

For  i  =  1  To  n 

'x(i)  values  must  be  one  column  to  the  left  of  cell  calling  function 
x(i)  =  Cells(row_first  +  i  - 1 ,  column  - 1 ) 

'y(i)  values  must  be  two  columns  to  the  left  of  cell  calling  function 
y(i)  =  Cells(row_first  +  i  -  1,  column  -  2) 
y_sort(i)  =  y(i) 

Next  i 

'if  there  are  less  than  3  data  points  EAC  cannot  be  calculated  so  exit  the  function 
If  n  <  3  Then 

RayleighTG  =  "" 

Exit  Function 
End  If 

'if  dACWP  at  this  point  is  <=  0  [i.e.,  y(n)  =  99999]  EAC  cannot  be  calculated  so 
'exit  the  function 
If  y(n)  =  99999  Then 
RayleighTG  = 

Exit  Function 
End  If 

'filter  out  any  data  points  where  dACWP  is  <=  0  [y(i)  =  99999] 

For  h  =  1  To  n  -  1 
For  i  =  h  To  n  -  1 
Ify(h)  =  99999  Then 
For  j  =  h  To  n  -  1 
x(j)  =  x(j  +  1) 

yG)  =  yO  + 1) 

y_sort(j)  =  y_sort(j  +  1) 

Next  j 
n  =  n  -  1 
End  If 
Next  i 
Next  h 

'if  there  are  now  less  than  3  data  points  EAC  cannot  be  calculated  so  exit  the  function 
If  n  <  3  Then 
Rayleigh_TG  =  "" 

Exit  Function 
End  If 

'calculate  the  base  number  of  points  in  each  group  (k)  and  the  remainder  (m) 
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k  =  Int(n  /  3) 
m  =  n  Mod  3 

'Sets  the  number  of  points  in  each  group  (L)eft,  (M)iddle,  (R)ight 
If  m  =  0  Then 
k_L  =  k 
k_M  =  k 
k_R  =  k 

Elself  m  =  1  Then 
k_L  =  k 
k_M  =  k  +  1 
k_R  =  k 
Else 

k_L  =  k  +  1 
k_M  =  k 
k_R  =  k  +  1 
End  If 

'place  (L)eft  group  y  values  in  ascending  order 
For  i  =  1  To  k_L  -  1 
For  j  =  i  +  1  To  k_L 

If  y_sort(j)  <  y_sort(i)  Then 
ytemp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
y_sort(j)  =  ytemp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (L)eft  group 
If  k_L  Mod  2  =  0  Then 
x_L  =  (x(k_L  /  2)  +  x((k_L  /  2)  +  1 ))  /  2 
y_L  =  (y_sort(k_L  /  2)  +  y_sort((k_L  /  2)  +  1 ))  /  2 
Elself  k_L  Mod  2  =  1  Then 
x_L  =  x(Int(k_L  /  2)  +  1 ) 
y_L  =  y_sort(Int(k_L  /  2)  +  1) 

End  If 

'place  (M)iddle  group  y  values  in  ascending  order 
For  i  =  k_L  +  1  To  k_L  +  k_M  - 1 
For  j  =  i  +  1  To  k_L  +  k_M 
If  y_sort(j)  <  y_sort(i)  Then 
ytemp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
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y_sort(j)  =  ytemp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (M)iddle  group 
Ifk_M  Mod  2  =  0  Then 

x_M  =  (x(k_L  +  (k_M  /  2))  +  x(k_L  +  (k_M  /  2)  +  1 ))  /  2 
y_M  =  (y_sort(k_L  +  (k_M  /  2))  +  y_sort(k_L  +  (k_M  /  2)  +  1 ))  /  2 
Elself  k_M  Mod  2  =  1  Then 
x_M  =  x(k_L  +  Int(k_M  /  2)  +  1) 
y_M  =  y_sort(k_L  +  Int(k_M  /  2)  +  1) 

End  If 

'place  (R)ight  group  y  values  in  ascending  order 
For  i  =  k_L  +  k_M  +  1  To  n  -  1 
For  j  =  i  +  1  To  n 

If  y_sort(j)  <  y_sort(i)  Then 
ytemp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
y_sort(j)  =  ytemp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (R)ight  group 
If  k_R  Mod  2  =  0  Then 

x_R  =  (x(k_L  +  k_M  +  (k_R  /  2))  +  x(k_L  +  k_M  +  (k_R  /  2)  +  1))  /  2 
y_R  =  (y_sort(k_L  +  k_M  +  (k_R  /  2))  +  y_sort(k_L  +  k_M  +  (k_R  /  2)  +  1))  /  2 
Elself  k_R  Mod  2  =  1  Then 
x_R  =  x(k_L  +  k_M  +  Int(k_R  /  2)  +  1) 
y_R  =  y_sort(k_L  +  k_M  +  Int(k_R  /  2)  +  1) 

End  If 

'calculate  initial  slope  [a(0)]  and  intercept  [b(0)] 
a(0)  =  (y_R  -  y_L)  /  (x_R  -  x_L) 

b(0)  =  ((y_L  -  a(0)  *  x_L)  +  (y_M  -  a(0)  *  x_M)  +  (y_R  -  a(0)  *  x_R))  /  3 

'initialize  Slope  and  Intercept  values 
Slope  =  0 
Intercept  =  0 

'iteratively  calculate  residuals  to  refine  Slope  and  Intercept  estimates 
'loop  executes  50  times  or  until  adjustment  is  <=  0.01%  of  slope 
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For  h  =  1  To  50 

Slope  =  Slope  +  a(h  -  1) 

Intercept  =  Intercept  +  b(h  - 1 ) 

'calculate  residuals  and  assign  values  to  y(i)  and  y_sort(i) 

For  i  =  1  To  n 

residual(h,  i)  =  y(i)  -  (b(h  -  1)  +  a(h  -  1)  *  (x(i))) 
y(i)  =  residual(h,  i) 
y_sort(i)  =  y(i) 

Next  i 

'place  (L)efit  group  y  (residual)  values  in  ascending  order 
For  i  =  1  To  k_L  -  1 
For  j  =  i  +  1  To  k_L 

If  y_sort(j)  <  y_sort(i)  Then 
y_temp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
y_sort(j)  =  y_temp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (L)eft  group  using  residuals  for  y  values 
If  k_L  Mod  2  =  0  Then 

x_L  =  (x(k_L  /  2)  +  x((k_L  /  2)  +  1 ))  /  2 
y_L  =  (y_sort(k_L  /  2)  +  y_sort((k_L  /  2)  +  1))  /  2 
Elself  k_L  Mod  2  =  1  Then 
x_L  =  x(Int(k_L  /  2)  +  1) 
y_L  =  y_sort(Int(k_L  /  2)  +  1) 

End  If 

'place  (M)iddle  group  y  (residual)  values  in  ascending  order 
For  i  =  k_L  +  1  To  k_L  +  k_M  - 1 
For  j  =  i  +  1  To  k_L  +  k_M 
If  y_sort(j)  <  y_sort(i)  Then 
ytemp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
y_sort(j)  =  ytemp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (M)iddle  group  using  residuals  for  y  values 
If  k_M  Mod  2  =  0  Then 
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x_M  =  (x(k_L  +  (k_M  /  2))  +  x(k_L  +  (k_M  /  2)  +  1))  /  2 
y_M  =  (y_sort(k_L  +  (k_M  /  2))  +  y_sort(k_L  +  (k_M  /  2)  +  1))  /  2 
Elself  k_M  Mod  2  =  1  Then 
x_M  =  x(k_L  +  Int(k_M  /  2)  +  1) 
y_M  =  y_sort(k_L  +  Int(k_M  /  2)  +  1) 

End  If 

'place  (R)ight  group  y  (residual)  values  in  ascending  order 
For  i  =  k_L  +  k_M  +  1  To  n  -  1 
For  j  =  i  +  1  To  n 

If  y_sort(j)  <  y_sort(i)  Then 
ytemp  =  y_sort(i) 
y_sort(i)  =  y_sort(j) 
y_sort(j)  =  ytemp 
End  If 
Next  j 
Next  i 

'calculate  median  values  for  (R)ight  group  using  residuals  for  y  values 
If  k_R  Mod  2  =  0  Then 

x_R  =  (x(k_L  +  k_M  +  (k_R  /  2))  +  x(k_L  +  k_M  +  (k_R  /  2)  +  1))  /  2 
y_R  =  (y_sort(k_L  +  k_M  +  (k_R  /  2))  +  y_sort(k_L  +  k_M  +  (k_R  /  2)  +  1))  /  2 
Elself  k_R  Mod  2  =  1  Then 

x_R  =  x(k_L  +  k_M  +  Int(k_R  /  2)  +  1 ) 
y_R  =  y_sort(k_L  +  k_M  +  Int(k_R  /  2)  +  1) 

End  If 

'calculate  slope  [a(h>]  and  intercept  [b(h)] 
a(h)  =  (y_R  -  y_L)  /  (x_R  -  x_L) 

b(h)  =  ((y_L  -  a(0)  *  x_L)  +  (y_M  -  a(0)  *  x_M)  +  (y_R  -  a(0)  *  x_R))  /  3 

'if  adjustment  value  [a(h)j  results  in  less  than  a  0.01%  change  then  Slope  is 
'sufficiently  accurate  so  exit  for  loop 
If  a(h)  /  Slope  <=  0.0001  Then 
Exit  For 
End  If 
Next  h 

'calculate  Rayleigh  parameters  (a,K)  and  EAC  (where  EAC=K) 

Rayleigha  =  -Slope 

Rayleigh  K  =  Exp(Intercept)  /  (-2  *  Slope) 

RayleighTG  =  RayleighK 

End  Function 


119 


Appendix  E:  Routine  Used  to  Calculate  Rayleigh  MMAE  EACs 


This  Appendix  contains  the  original  Visual  Basic®  routine,  provided  by  Capt  Mark 
Gallagher,  used  to  calculate  the  Rayleigh  MMAE  EACs.  Modifications  were  made  to  the 
original  code  to  allow  the  use  of  the  SAF/FMCE  inflation  indices  that  were  used  to 
calculate  the  inflation  adjusted  EACs  for  the  other  EAC  methods.  These  modifications 
have  been  noted  with  comments. 


i***:M*********:M******************************************************* 

Option  Explicit 
Option  Base  1 


Const  MaxData  =100  '  Maximum  data  points  process  by  procedure 

Const  Numd  =  20  '  Number  of  cost  parameters  considered 

Const  Num  Alpha  =  20  '  Number  of  time  parameters  considered 

Const  NumK  =  5  '  Number  of  Kalman  filter  gains  applied 

Const  NumFilters  =  Numd  *  NumAlpha  *  NumK  '  Number  of  Kalman  filters  applied 


Dim  lnflaters(0  To  64)  As  Single  '  Navy  RDT&E  inflators  from  1960  (index  0)  to 
2024 

Dim  Alpha Array(Num Alpha,  2)  As  Single '  Alpha  values  and  constant-to-current  dollar 
conversions 


Dim  StartDay  As  Date  '  Program  start  date 

Dim  NumData  As  Integer  ’  Number  of  cost  reports  (data  points) 

Dim  ACWP(0  To  MaxData,  2)  As  Single  ’  First  is  time  index  and  Second 

'  is  Actual  Cost  of  Work  Performed  (ACWP)  in  base-year  dollars 

I*********************************************************************** 

'This  code  has  been  added  to  allow  the  use  of  the  SAF/FMCE  inflation  indices  that 
'were  used  for  the  other  EAC  Methods 
Dim  row  As  Integer 

l***************************^****************************************** 


Dim  MinTime  As  Single  '  Minimum  time  for  program  completion 
Dim  MaxTime  As  Single  '  Maximum  time  for  program  completion 
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Dim  MinAlpha  As  Single  '  Alpha  parameter  for  minimum  time  (MinTime,  but 
Largest  Alpha) 

Dim  MaxAlpha  As  Single  '  Alpha  parameter  for  maximum  time  (MaxTime,  but 
smallest  Alpha) 

Dim  MinCost  As  Single  '  Minimum  cost  at  program  completion 
Dim  MaxCost  As  Single  ’  Maximum  cost  at  program  completion 
Dim  Mind  As  Single  '  Minimum  parameter  d 

Dim  Maxd  As  Single  '  Maximum  parameter  d 

Dim  ResidVar  As  Single  '  Kalman  filter  residual  variance 
Dim  Resid  As  Single  '  Kalman  filter  residual 

Dim  Prob  As  Single  ’  Filter  probability  (Leading  term  cancels  in  normalization) 
Dim  ProbSum  As  Single  '  Sum  of  filter  probabilities 

Dim  MMAED  As  Single  '  MMAE  probabilistic  weighted  estimate  for  final  cost 
Dim  MMAEDTY  As  Single  '  MMAE  probabilistic  weighted  final  cost  in  current 
dollars 

Dim  MMAETime  As  Single  '  MMAE  probabilistic  weighted  estimate  for  final  time 

Function  MyTrunc(Value  As  Single)  As  Integer 
'  Truncates  a  real  number  to  an  integer 
If  (Value  >=  0)  Then 
MyTrunc  =  Value  -  Decimal(Value) 

Else 

MyTrunc  =  Value  +  Decimal(Value) 

End  If 

End  Function 

Function  Decimal(Value  As  Single)  As  Single 
'  Returns  the  decimal  portion  of  a  real  number 
If  (((Value  -  Int(Value))  >=  0)  And  (Value  >  0))  Then  '  rounded  down 
Decimal  =  Value  -  Int(Value) 

Elself  (((Value  -  Int(Value))  <  0)  And  (Value  >  0))  Then  '  rounded  up 
Decimal  =  Value  -  Int( Value  - 1) 

Elself  (((Value  -  Int(Value))  >=  0)  And  (Value  <  0))  Then  '  rounded  down 
Decimal  =  Int(Value  +  1 )  -  Value 

Elself  (((Value  -  Int(Value))  <  0)  And  (Value  <  0))  Then  '  rounded  up 
Decimal  =  Int(Value)  -  Value 
Else 

Decimal  =  0 
End  If 

End  Function 
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Function  Translator(Start  As  Date,  alpha  As  Single)  As  Single 
'  Determines  the  translation  factor  from  a  constant  dollar  Rayleigh  expenditure 
'  profile  to  a  current  dollar  total 
Dim  Duration  As  Single 
Dim  Time  As  Single 
Dim  LastTime  As  Single 
Dim  Yrlndex  As  Integer 
Dim  Sum  As  Single 

'  Initial  '(call  to  Procedure  Initial  needed  if  not  called  from  MMAECost ) 

Duration  =  Sqr(3.5  /  alpha) 

Sum  =  0 

Yrlndex  =  MyTrunc((Start  -  #10/1/59#)  /  365.25) 

LastTime  =  0 

Time  =  Decimal((Start  -  #10/1/59#)  /  365.25) 

While  (Time  <  Duration) 

Sum  =  Sum  +  Inflaters( Yrlndex)  *  (Exp(-1  *  alpha  *  LastTime  *  LastTime)  -  Exp(-1 
*  alpha  *  Time  *  Time)) 

LastTime  =  Time 
Time  =  Time  +  1 
Yrlndex  =  Yrlndex  +  1 
Wend 

Sum  =  Sum  +  Inflaters( Yrlndex)  *  (Exp(-1  *  alpha  *  LastTime  *  LastTime)  -  Exp(-1  * 
alpha  *  Duration  *  Duration)) 

Translator  -  Sum 
End  Function 

Sub  Initial(row) 

Dim  I  As  Integer 

1****** ******************************************************  *********** 

'This  section  of  code  has  been  substituted  to  allow  the  use  of  the  SAF/FMCE 
'inflation  indices  that  were  used  for  the  other  EAC  methods 
'Note  that  the  inflation  indices  for  development  contracts  (3600)  must  be  located  in 
'column  25  (Y)  of  the  spreadsheet,  and  the  indices  for  production  contracts  (3080)  must 
'be  located  in  column  26  (Z)  of  the  spreadsheet.  The  inflation  indices  for  1960  are  located 
'in  row  13. 

For  I  =  0  To  64 

If  Cells(row,  9)  =  "D"  Then 
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Inflaters(I)  =  Cells(I  +  13,  25) 

Else 

Inflaters(I)  =  Cells(I  +  13, 26) 

End  If 
Next  I 

'*********************************************************************** 


This  section  has  been  commented  out  of  the  original  code  to  allow  use  of  the 
'SAF/FMCE  inflation  indices  that  were  used  for  the  other  EAC  methods 


'lnflaters(O)  =  0.2108 ' 1960  Navy  Raw  RDT&E  Inflation  Indice  (NCA  Mar  1994) 

’Inflaters(l)  =  0.2131 

'Inflaters(2)  =  0.2161 

’Inflaters(3)  =  0.2198 

’lnflaters(4)  =  0.223 

’Inflaters(5)  =  0.2271 

’Inflaters(6)  =  0.2332 

’Inflaters(7)  =  0.2407 

’Inflaters(8)  =  0.2494 

'Inflaters(9)  =  0.2612 

'lnflaters(lO)  =  0.2772 ' 1970  Navy  Raw  RDT&E  Inflation  Indice  (NCA  Mar  1995) 

'Inflaters(  11)  =  0.29 1 5 

'Inflaters(12)  =  0.3049 

'Inflaters(  1 3)  =  0.3 1 82 

’Inflaters(  1 4)  =  0.3435 

*Inflaters(  1 5)  =  0.381 1 

’Inflaters(16)  =  0.4063 

'Inflaters(17)  =  0.4288 

'Inflaters(18)  =  0.458 

'Inflaters(  1 9)  =  0.4964 

'Inflaters(20)  =  0.549 ' 1980  Navy  Raw  RDT&E  Inflation  Indice  (NCA  Mar  1995) 

Tnflaters(21)  =  0.6073 

’lnflaters(22)  =  0.6534 

’Inflaters(23)  =  0.6854 

’Inflaters(24)  —  0.71 1 5 

’Inflaters(25)  =  0.7357 

’Inflaters(26)  =  0.7563 

’Inflaters(27)  =  0.7767 

’Inflaters(28)  =  0.8 

'Inflaters(29)  =  0.8336 

’Inflaters(30)  =  0.8669 ' 1990  Navy  Raw  RDT&E  Inflation  Indice  (NCA  Mar  1995) 
'Inflaters(31)  =  0.9042 
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’Inflaters(32)  =  0.9295 
'Inflaters(33)  =  0.9546 
'Inflaters(34)  =  0.9737 
'Inflaters(35)  =  1 
'Inflaters(36)  =  1.03 

Tor  I  =  37  To  64 

'lnflaters(I)  =  Inflaters(I  -  1)  *  1.03 
Tfext  I 

'****************************4 <****************************************** 


End  Sub 


Sub  Kalman(d  As  Single,  alpha  As  Single,  gain  As  Single,  sumsqresids  As  Single) 

Dim  XMinus  As  Single 
Dim  XPlus  As  Single 
Dim  1  As  Integer 

sumsqresids  =  0 

’  Propagate  through  time  from  initial  time  and  no  cost  -  Eq(  7) 

XMinus  =  d  *  (1  -  Exp(-1  *  alpha  *  (ACWP(1, 1))  A  2)) 

Resid  =  AC WP(  1 , 2)  -  XMinus 

sumsqresids  =  sumsqresids  +  Resid  *  Resid 

'  First  update  state  mean  distribution  estimate  -  Eq  (9) 

XPlus  =  XMinus  +  gain  *  Resid 

’  Check  that  in  Visual  Basic  LOG  is  the  natural  logarithm  function  -  It  is! 

For  I  =  2  To  NumData '  for  each  datum 
'  Propagate  through  time  -  Eq(  7) 

XMinus  =  d  *  (1  -  Exp(-1  *  alpha  *  (ACWP(1, 1)  -  ACWP(I  - 1, 1)  +  ((-1  /  alpha)  * 
Log(l  -  (XPlus  /  d))) A  0.5) A  2)) 

'  Determine  residual  -  Eq  (8) 

Resid  -  ACWP(I,  2)  -  XMinus 
sumsqresids  =  sumsqresids  +  Resid  *  Resid 
'  Update  state  mean  distribution  estimate  -  Eq  (9) 

XPlus  =  XMinus  +  gain  *  Resid 
Next  I 
End  Sub 

Sub  Algorithm!} 

Dim  I  As  Integer  '  Cost  parameter  (d)index 
Dim  J  As  Integer  '  Time  parameter  (Alpha)  index 
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Dim  K  As  Integer  '  Kalman  filter  gain  (k)  index 
Dim  L  As  Integer  '  Filter  counter 

Dim  Filter(NumFilters,  5)  As  Single 
'  1  is  parameter  d 
'  2  is  parameter  Alpha 
'  3  is  Kalman  filter  gain 

'  4  is  sum  residuals  squared  and  filter  probability  for  second  pass 
'  5  is  the  constant-to-current  dollar  converter 
Dim  XHat(NumFilters)  As  Single  '  Filter  state  estimates 

Dim  DeltaD  As  Single  '  Delta  between  filters'  parameters  d 
Dim  DeltaAlpha  As  Single  '  Delta  between  filters'  parameters  Alpha 
Dim  Temp  As  Single  '  Temporary  variable 

■  *****  STEP  2  ***** 

'  Set  defaults  for  time  range 
MinTime  =  ACWP(NumData,  1) 

MaxTime  =15 

'  Query  analyst  to  revise  default  final-time  range 

MinAlpha  =  3.5  /  (MinTime  *  MinTime)  '  Eq  (4) 

Max  Alpha  =  3.5  /  (MaxTime  *  MaxTime)  '  Eq  (4) 

'  Set  defaults  for  cost  range 
MinCost  =  ACWP(NumData,  2) 

MaxCost  =  ACWP(NumData,  2)  /  (1  -  Exp(-1  *  MaxAlpha  *  ACWP(NumData,  1)  * 
ACWP(NumData,  1))) '  Eq  (1 1) 

'  Query  analyst  to  revise  default  final-cost  range 

Mind  =  MinCost  7  0.97  '  Eq  (3) 

Maxd  =  MaxCost  7  0.97  '  Eq  (3) 


■  *****  sjep  3  ***** 


DeltaD  =  (Maxd  -  Mind)  /  (Numd  - 1) 

DeltaAlpha  =  (MaxAlpha  -  MinAlpha)  /  (NumAlpha  - 1)  ’  negative  number  since 
MinAlpha  >  MaxAlpha 


L  =  0 
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For  I  =  1  To  Numd 
For  J  =  1  To  NumAlpha 
For  K  =  1  To  NumK 
L  =  L  +  1 

Filter(L,  1)  =  Mind  +  (I  - 1)  *  DeltaD 
Filter(L,  2)  =  MinAlpha  +  (J  -  1)  *  DeltaAlpha 
Filter(L,  3)  =  (K  -  1)  /NumK 
Filter(L,  4)  =  0 
Next  K 
Next  J 
Next  I 

»  *****  gygp  5  ***** 

Kalman  Filter(l,  1),  Filter(l,  2),  Filter(l,  3),  Filter(l,  4) 

ResidVar  =  Filter(l,  4) 

For  L  =  2  To  NumFilters 

Kalman  Filter(L,  1),  Filter(L,  2),  Filter(L,  3),  Filter(L,  4) 

If  (Filter(L,  4)  <  ResidVar)  Then 
ResidVar  =  Filter(L,  4) 

End  If 
Next  L 

'  Reduce  parameter  range 

Temp  =  Mind 
Mind  =  Maxd 
Maxd  =  Temp 
Temp  =  MinAlpha 
MinAlpha  =  MaxAlpha 
Max  Alpha  =  Temp 

For  L  =  1  To  NumFilters 

If  (Filter(L,  4)  <=  3  *  ResidVar)  Then '  Retain  parameters  in  reduced  parameter  range 
If  (Mind  >  Filter(L,  1))  Then 
Mind  =  Filter(L,  1) 

End  If 

If  (Maxd  <  Filter(L,  1))  Then 
Maxd  =  Filter(L,  1) 

End  If 

If  (MinAlpha  <  Filter(L,  2))  Then 
MinAlpha  =  Filter(L,  2) 

End  If 
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If  (MaxAlpha  >  Filter(L,  2))  Then 
MaxAlpha  =  Filter(L,  2) 

End  If 
End  If 
Next  L 


MaxTime  -  (3.5  /  MaxAlpha)  A  0.5  1  Relationship  in  Eq  (4) 
MinTime  =  (3.5  /  MinAlpha)  A  0.5  '  Relationship  in  Eq  (4) 


ResidVar  =  (1  /  (NumData  - 1))  *  ResidVar 


'  Equally  space  filter  parameters  across  reduced  range  and 
'  Reset  prior  mens  and  set  filter  probabilities 

DeltaD  =  (Maxd  -  Mind)  /  (Numd  - 1 ) 

DeltaAlpha  -  (MaxAlpha  -  MinAlpha)  /  (NumAlpha  - 1)  ’  negative  number  since 
MinAlpha  >  MaxAlpha 

L  =  0 

For  I  =  1  To  Numd 
For  J  =  1  To  NumAlpha 
For  K  =  1  To  NumK 
L  =  L  +  1 

Filter(L,  1)  =  Mind  +  (I  -  1)  *  DeltaD 
Filter(L,  2)  =  MinAlpha  +  (J  - 1)  *  DeltaAlpha 
Filter(L,  3)  =  (K  -  1)  /NumK 

XHat(L)  =  0  'set  prior  state  distribution  means  for  each  filter  to  zero 
Filter(L,  4)  =  1  /  NumFilters  '  set  filter  prior  probabilities 
Filter(L,  5)  =  Translator(StartDay,  Filter(L,  2)) 

Next  K 
Next  J 
Next  I 

i  *****  step  6  ***** 

'  First  data  point  through  all  the  filters 

ProbSum  =  0 

For  L  =  1  To  NumFilters 

XHat(L)  =  Filter(L,  1)  *  (1  -  Exp(-1  *  Filter(L,  2)  *  (ACWP(1,  1))  A  2)) 

Resid  =  ACWP(1,  2)  -  XHat(L) 

'  First  update  state  mean  distribution  estimate  -  Eq  (9) 
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XHat(L)  -  XHat(L)  +  Filter(L,  3)  *  Resid 
Prob  =  Exp(-1  *  Resid  *  Resid  /  (2  *  ResidVar)) 
Filter(L,  4)  =  Filter(L,  4)  *  Prob 
ProbSum  =  ProbSum  +  Filter(L,  4) 

Next  L 


For  L  -  1  To  NumFilters 
Filter(L,  4)  =  Filter(L,  4)  /  ProbSum 
If  (Filter(L,  4)  <  0.001)  Then 
Filter(L,  4)  =  0.001 
End  If 
Next  L 

For  I  -  2  To  NumData '  for  each  datum  after  the  first  datum 
ProbSum  =  0 

For  L  =  1  To  NumFilters  '  for  each  Kalmam  filter 
'  Propagate  through  time  -  Eq(  7) 

If  (XHat(L)  >  Filter(L,  1))  Then 
XHat(L)  =  Filter(L,  1) 

Else 

XHat(L)  =  Filter(L,  1)  *  (1  -  Exp(-1  *  Filter(L,  2)  *  (ACWP(I,  1)  -  ACWP(I  -  1  1) 
+  ((-1  /  Filter(L,  2))  *  Log(l  -  (XHat(L)  /  Filter(L,  1))))  A  0.5)  A  2)) 

End  If 

'  Determine  residual  -  Eq  (8) 

Resid  =  ACWP(I,  2)  -  XHat(L) 

'  Update  state  mean  distribution  estimate  -  Eq  (9) 

XHat(L)  =  XHat(L)  +  Filter(L,  3)  *  Resid 
Prob  =  Exp(-1  *  Resid  *  Resid  /  (2  *  ResidVar)) 

Filter(L,  4)  =  Fiiter(L,  4)  *  Prob 
ProbSum  =  ProbSum  +  Filter(L,  4) 

Next  L 

For  L  =  1  To  NumFilters 
Filter(L,  4)  =  Filter(L,  4)  /  ProbSum 
If  ((Filter(L,  4)  <  0.001)  And  (I  <  NumData))  Then 
Filter(L,  4)  =  0.001 
End  If 
Next  L 
Next  I 

'  *****  STEP  7  ***** 

MMAED  =  0 
MMAETime  =  0 
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MMAEDTY  =  0  '  Initial  current  (then-year)  summation  term 
For  L  =  1  To  NumFilters 

Temp  =  0.97  *  Filter(L,  1)  '  Relationship  from  Eq  (3) 

If  (Temp  <  ACWP(NumData,  2))  Then 
Temp  =  ACWP(NumData,  2) 

End  If 

MMAED  =  MMAED  +  Temp  *  Filter(L,  4) 

MMAETime  =  MMAETime  +  Filter(L,  2)  *  Filter(L,  4) 

MMAEDTY  =  MMAEDTY  +  Filter(L,  1)  *  Filter(L,  4)  *  Filter(L,  5) 
Next  L 

MMAETime  =  (3.5  /  MMAETime)  A  0.5  '  Relationship  from  Eq  (4) 

L  =  0 

For  I  =  1  To  Numd 
For  J  =  1  To  NumAlpha 
ProbSum  =  0 
For  K  =  1  To  NumK 
L  =  L  +  1 

ProbSum  =  ProbSum  +  Filter(L,  4) 

Next  K 
Next  J 
Next  I 

For  J  =  1  To  NumAlpha 
1=1  +(J-  l)*NumK 
Next  J 
End  Sub 


Function  MMAECost(StartDate,  CumCostRpts,  RptDates,  row) 

'  See  QUE  Excel  version  5  Book  on  page  1234  for  an  example 

'  CumCostRpts  is  either  numbers  or  references  to  cumulative  current  (Then- Year)  dollar 
expenditures 

1  RptDates  is  either  cost  report  dates  or  references  to  the  dates  of  cost  data  reports 
'************************************************************+****+***** 

'The  additional  parameter,  row,  has  been  added  to  allow  use  of  the  S  AF/FMCE 

'inflation  indices  that  were  used  for  the  other  EAC  methods 

***********  **********************************************  *************** 


Dim  I  As  Integer 
Dim  Imin  As  Integer 
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Dim  Imax  As  Integer 
Dim  J  As  Integer 
Dim  RptDate  As  Date 
Dim  Tempi  As  Integer 
Dim  Temp  As  Single 

Initial  (row)  '  Initializes  inflation  indices 

If  TypeName(StartDate)  =  "range"  Then 
StartDay  =  StartDate.  Value 
Else 

StartDay  =  StartDate 
End  If 

ACWP(0,  1)  =  0  '  The  program  starts  at  time  index  of  zero 
ACWP(0,  2)  —  0  The  program  starts  with  no  expenditures 


'  See  what  is  being  passed  in  RptDates 

If  it  is  a  Range  object,  transfer  the  data  in  the  reference  cells  to  the  ACWP  array. 

If  TypeName(RptDates)  =  "Range"  Then 
If  (RptDates. Columns. Count =  1)  Then  '  data  are  in  a  column 
'  Find  the  number  of  rows 
NumData  =  RptDates.Rows.Count 

'  Adjust  dates  to  program  time  indices 
For  I  =  1  To  NumData 

ACWP(I,  1)  =  (RptDates.Cells(I,  1). Value  -  StartDay)  /  365.25 
Next  I 

Elself  (RptDates.Rows.Count =  1)  Then  '  data  are  in  a  row 
'  Find  the  number  of  columns 
NumData  =  RptDates.Columns. Count 

’  Copy  the  data  from  the  range  into  the  array 
For  I  =  1  To  NumData 

ACWP(I,  1)  =  (RptDates.Cells(l,  I). Value  -  StartDay)  /  365.25 
Next  I 

Elself  ((RptDates.Columns.Count >  1)  And  (RptDates.Rows.Count  >  l))Then  'error 
MMAECost  =  Array(CVErr(xlErrValue)) 

Exit  Function 
End  If 

Elself  (VarType(RptDates)  >=  8192)  Then  'an  array  so  return  an  error  and  quit 
Imin  =  LBound(RptDates,  1) 
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Imax  =  UBound(RptDates,  1) 

NumData  =  Imax  -  Imin  +  1 
J=  1 

For  I  =  Imin  To  Imax 
ACWP(J,  1)  =  RptDates(I) 

J  =  J+  1 
Next  I 

Elself  (VarType(RptDates)  <8192)  Then  'not  an  array  so  return  an  error  and  quit 
MMAECost  =  Array(CVErr(xlErrValue)) 

Exit  Function 
End  If 


'  See  what  is  being  passed  in  CumCostRpts 

'  If  it  is  a  Range  object,  transfer  the  data  in  the  reference  cells  to  the  ACWP  array. 

If  TypeName(CumCostRpts)  =  "Range"  Then 
If  (CumCostRpts.Columns.Count  =  1)  Then  ’  data  are  in  a  column 
'  Find  the  number  of  rows 
NumData  =  CumCostRpts.Rows.Count 

'  Copy  the  data  from  the  range  into  the  array 
For  I  =  1  To  NumData 
RptDate  =  StartDay  +  ACWP(I,  1)  *  365.25 

Tempi  =  MyTrunc((RptDate  -  #3/1/60#)  /  365.25) '  Inflater  index  for  mid-fiscal 

year 

Temp  =  Decimal((RptDate  -  #3/1/60#)  /  365.23)  '  Interpolation  between  annual 
indices 

Temp  =  Inflaters(TempI)  +  Temp  *  (Inflaters(TempI  +  1)  -  Inflaters(TempI)) 

If  (I  =  1)  Then 

ACWP(1,  2)  =  CumCostRpts.Cells(l ,  l).Value  /  Temp 
Else 

ACWP(I,  2)  =  ACWP(I  -1,2)  +  (CumCostRpts.Cells(1, 1). Value  - 
CumCostRpts.Cells(I  - 1, 1).  Value)  /  Temp 
End  If 
Next  I 

Elself  (CumCostRpts.Rows.Count  =  1)  Then  ’  data  are  in  a  row 
'  Find  the  number  of  columns 
NumData  =  CumCostRpts.Columns.Count 

'  Copy  the  data  from  the  range  into  the  array 
For  I  =  1  To  NumData 
RptDate  =  StartDay  +  ACWP(I,  1)  *  365.25 
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Tempi  MyTrunc((RptDate  -  #3/1/60#)  /  365.25)  '  Inflater  index  for  mid-fiscal 

year 

Temp  —  Decimal((RptDate  -  #3/1/60#)  /  365.23)  '  Interpolation  between  annual 
indices 

Temp  —  Inflaters(TempI)  +  Temp  *  (Inflaters(TempI  +  1)  -  Inflaters(TempI)) 

If  (I  =  1 )  Then 

ACWP(1,  2)  =  CumCostRpts.Cells(  1 ,  1). Value  /  Temp 
Else 

ACWP(I,  2)  =  ACWP(I  -1,2)  +  (CumCostRpts.Cells(  1 , 1).Value  - 
CumCostRpts.Cells(  1,1-1 ).  Value)  /  Temp 
End  If 
Next  I 

Elself  ((CumCostRpts.Columns.Count  >  1)  And  (CumCostRpts.Rows.Count  >  1)) 
Then  '  error 

MMAECost  =  Array(CVErr(xlErrValue)) 

Exit  Function 
End  If 

Elself  (V arType(CumCostRpts)  >=  8 1 92)  Then  'an  array 
Imin  =  LBound(CumCostRpts,  1) 

Imax  =  UBound(CumCostRpts,  1) 

NumData  =  Imax  -  Imin  +  1 
J  =  1 

For  I  =  Imin  To  Imax 

RptDate  =  StartDay  +  ACWP(I,  1)  *  365.25 

Tempi  =  MyTrunc((RptDate  -  #3/1/60#)  /  365.25) '  Inflater  index  for  mid-fiscal 

year 

Temp  =  Decimal((RptDate  -  #3/1/60#)  /  365.23)  '  Interpolation  between  annual 
indices 

Temp  =  Inflaters(TempI)  +  Temp  *  (Inflaters(TempI  +  1)  -  Inflaters(TempI)) 

If  (J  =  1 )  Then 

ACWP(1,  2)  =  CumCostRpts(I)  /  Temp 
Else 

ACWP(J,  2)  =  ACWP(J  -1,2)  +  (CumCostRpts(I)  -  CumCostRpts(I  - 1))  /  Temp 
End  If 
J  =  J  +  1 
Next  I 

Elself  (VarType(CumCostRpts)  <  8192)  Then  'not  an  array  so  return  an  error  and  quit 
MMAECost  =  Array(CVErr(xlErrValue)) 

Exit  Function 
End  If 
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Algorithm 

MMAECost  =  MMAEDTY 


End  Function 


Appendix  F:  Routine  Used  to  Calculate  Beta  EACs 


**************^**t ***************************************************** 

UDF  Name:  BetaEAC 
Written  by:  Capt  Todd  D.  Nystrom 
Date:  3  May  1995 

This  Microsoft®  Excel  user  defined  function  (UDF)  calculates  an  EAC  using  the 
modified  Beta  function  described  in  Whitlock,  1982.  The  modified  Beta  function 
(Equation  [19])  expresses  cumulative  fractional  cost  as  a  function  of  fractional  time  and 
is  given  by  the  following: 

C(t)=A(10tA2  -  20tA3  +  1 0tA4)  +  B(10tA3  -  20tA4  +  10tA5)  +  (5tA4  -  4tA5) 
subject  to  the  constraints: 

A+B  <=  1 ,  A  >=  0,  B  >=  0 

The  routine  attempts  to  minimize  the  weighted  sum  of  least  squares  (WSLS) 

(Equation  [20])  given  by  the  following: 

WSLS  =  SUM[C(t(i))  -  Y  *  K(i)]A2  *  t(i)A3 

The  optimum  value  of  Y  (Y_opt)  is  calculated  for  given  values  of  A  and  B 
using  Equation  (21)  as  follows: 

Y  opt  =  SUM[C(t(i))  *  K(i)  *  t(i)A3]  /  SUM[t(i)A3  *  K(i)A2] 

A  and  B  are  first  tested  at .  1  increments,  if  a  given  combination  produces  a  lower  value 
for  WSLS  than  the  previous  values  of  Y,  A  and  B  then  the  old  values  are  replaced  and  A 
and  B  are  tested  at  .01  increments.  If  the  refined  values  result  in  a  lower  WSLS  the  old 
are  again  replaced  and  the  routine  moves  on  to  the  next  increments  of  A  and  B. 

Input  Variables 

row  first:  indicates  the  first  row  of  data  for  this  contract 

row  current:  indicates  the  row  number  of  cell  calling  this  function 

column:  indicates  the  column  number  of  cell  calling  this  function 

Note:  fractional  time  values  [t(i)]  must  be  located  one  column  to  the  left  of  the  cell 
calling  this  function,  and  ACWP  values  [K(i)]  must  be  located  six  columns  to  the  left 
of  the  cell  calling  this  function. 
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'Output:  the  modified  Beta  curve  EAC  [Equation  (24)]  is  output  directly  to  the  cell 
calling  this  function  and  is  given  by  the  following  equation: 

t 

'  EAC  =  1  /  Y 

f 

*********  *******************************  **************************** 

Function  Beta_EAC(row_first,  row  current,  column) 

'dimension  arrays 

Dim  t ( 6 0 )  As  Double  'array  for  fractional  time  values 
Dim  K(60)  As  Double  'array  for  ACWP  values 

'determine  number  of  data  points  (n)  for  the  contract  at  the  current  time 
n  --  row  current  -  row_first  +  1 

'place  fractional  t  values  in  t(i).  ACWP  values  in  K(i) 

For  i  =  1  To  n 

't(i)  values  must  be  one  column  to  the  left  of  cell  calling  function 
t(i  I  —  Cells) row  first  +  i  -  1 .  column  -  1 ) 

'ACWP  values  must  be  6  columns  to  the  left  of  cell  calling  function 
K(i)  =  Cel ls(ro\v_ first  -  i  -  1.  column  -  6) 

Next  i 


'initialize  WSLS  to  a  high  number  (because  WSLS  must  be  minimized) 

WSLS  =  999990999 

'test  values  of  A  and  B  at  .1  increments 
For  A_pick  =  0  To  1  Step  0.1 
For  B_pick  ~  0  To  1  Step  0. 1 

'if  A  -  B  1  solution  meets  constraints  so  check  for  min  WSLS 
If  A_pick  -  B_pick  <=  1  Then 

'initialize  numerator,  denominator,  and  Y_opt  to  0 
numerator  =  0 
denominator  =  0 
Y_opt  =  0 

'calculate  Y_opt  per  Equation  (21) 

For  i  =  1  To  n 

numerator  =  numerator  +  C_t(t(i).  A_pick.  B_pick)  *  K(i)  *  (t(i)  A  3) 
denominator  =  denominator  +  (K(i)  A  2)  *  (t(i)  A  3) 


Next  i 

Yopt  =  numerator  /  denominator 

'initialize  WSLStemp  to  0 
WSLS_temp  :=  0 

'calculate  WSLS  per  Equation  (20) 

For  i  =  1  To  n 

WSLS  temp  =  WSLS  temp  +  ((C _ t(t(i).  A_pick,  B_pick)  -  Y_opt  *  K(i))  A 

2)  *  (t(i)  A  3) 

Next  i 

'if  WSLS  temp  <  WSLS  then  Y  opt.  A _pick.  and  B_pick  are  a 
'better  solution  so  replace  old  values  and  check  for  better 
'solution  at  .01  increments 
If  WSLS  temp  <  WSLS  Then 
'replace  with  new  values 
y  =  Y  opt 
A  =  Apick 
B  =  B_pick 
WSLS  =  WSLS  temp 

'test  values  of  A  and  B  at  .01  increments  from  A  -  0.05 
'to  A  +  0.05  and  B  -  0.05  to  B  +  0.05 
For  A  refined  =  Ajick  -  0.05  To  A_pick  +  0.05  Step  0.01 
For  Brefined  =  B_pick  -  0.05  To  Bj>ick  +  0.05  Step  0.01 
'if  A  +  B  <=  1  solution  meets  constraints  so 
'check  for  min  WSLS 
If  A_refmed  +  B_re fined  <=  1  Then 

'initialize  numerator,  denominator,  and  Y  opt 
'to  0 

numerator  =  0 
denominator  =  0 
Y_opt  =  0 

'calculate  Y  opt  per  Equation  (21 ) 

For  i  =  1  To  n 

numerator  =  numerator  +  C_t(t(i),  A_refined,  B_refined)  *  K(i)  * 

(t(i)  A  3) 

denominator  =  denominator  +  (K(i)  A  2)  *  (t(i)  A  3) 

Next  i 

Y_opt  =  numerator  /  denominator 
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'initialize  WSLS_temp  to  0 
WSLStemp  =  0 

'calculate  WSLS  per  Equation  (20) 

For  i  =  1  To  n 

WSLS_temp  =  WSLS  temp  +  ((C_t(t(i),  Arefined.  B_refined)  - 
Y_opt  *  K(i))  A  2)  *  (t(i)  A  3) 

Next  i 

'if  WSLS  temp  <  WSLS  then  Y  opt,  A  refined, 

'and  B_re  fined  are  a  better  solution  so 

'replace  old  values 

If  WSLS  temp  <  WSLS  Then 

y  =  Y°pt 

A  =  A  refined 
B  =  B_re  fined 
WSLS  =  WSLSjemp 
End  If 
End  If 

Next  B  refined 
Next  Are  fined 
End  If 
End  If 
Next  B_pick 
Next  A_pick 

'calculate  Beta  EAC  per  Equation  (24) 

Beta  EAC  =  1  /  \ 

End  Function 


'this  function  calculates  C(t)  per  Equation  (19)  and  is  used  to  simplify  calculations 
'in  the  main  routine 

t*********************************************************************** 


Function  C_t(t_temp,  a_temp,  b_temp) 

C_t  =  atemp  *(10*  (tjemp  A  2)  -  20  *  (t_temp  A  3)  +  1 0  *  (t_temp  A  4))  +  b_temp  * 
(10*  (t  temp  A  3 )  -  20  *  (t  temp  A  4)  +  1 0  *  (t_temp  A  5))  +  (5  *  (t_temp  A  4)  -  4  * 
(t_temp  A  5)) 

End  Function 
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Appendix  G:  Raw  Inflation  Indices 


Year 

1960 

1961 

1962 

1963 

1964 

1965 

1966 

1967 

1968 

1969 

1970 

1971 

1972 

1973 

1974 

1975 

1976 

1977 

1978 

1979 

1980 

1981 

1982 

1983 

1984 

1985 

1986 

1987 

1988 

1989 

1990 

1991 

1992 

1993 

1994 

1995 


3600 

0 .208 
0.211 
0 . 213 
0 . 214 
0 .222 
0.230 
0.240 
0 . 258 
0 .272 
0 .277 
0.287 
0.300 
0.311 
0 . 325 
0 .361 
0 .382 
0.404 
0 .425 
0.449 
0 .485 
0 . 521 
0 . 646 
0 .685 
0 .714 
0 . 737 
0 . 758 
0 . 776 
0 .794 
0 .815 
0 . 843 
0 . 869 
0 .899 
0 . 931 
0 . 960 
0 . 978 
1 .000 


3080 

0 .205 
0 . 208 
0 .210 
0 .214 
0.217 
0.221 
0 .227 
0 . 234 
0 . 243 
0 .254 
0 . 268 
0.282 
0 . 295 
0.308 
0 .332 
0 .368 
0.393 
0.420 
0.448 
0 .487 
0.535 
0 . 598 
0 . 653 
0 . 685 
0 .711 
0 . 736 
0 . 756 
0 . 777 
0 .800 
0 . 834 
0 . 867 
0 . 904 
0 . 930 
0 . 955 
0 . 974 
1.000 
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Appendix  H:  Detailed  Results 


This  appendix  contains  the  detailed  results  for  the  performance  of  the  1 5  EAC 
methods  evaluated  in  this  study.  Each  sheet  contains  the  overall  results  for  each  EAC 
method  within  a  given  category  of  data  as  well  as  the  quartile  results  within  the  category. 
The  first  sheet  for  each  category  shows  the  results  for  the  non-inflation  adjusted  data 
(TV).  While  a  second  sheet  shows  the  results  for  the  inflation  adjusted  data  (CY95).  The 
data  shown  for  each  set  of  results  includes  the  MAPE  value,  the  SDAPE  value,  and  the 
combined  score  for  each  of  the  EAC  methods. 
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Results  for  All  Contracts  Using  Non-Inflation  Adjusted  Data 


1  TV  Results  for  Overall  Performance  | 

Score 

MAPF 

SDAPF 

CPI  5 

14 

1  2  9% 

19.3% 

CPI6 

4 

1  1  4% 

14  6% 

C  Pie  urn 

6 

1  1  6% 

1 4  9% 

SIM  3 

17 

1 4  7% 

23  1% 

SPI6 

8 

12  5% 

16.5% 

SPIcum 

10 

12.5% 

17  7% 

SCI  3 

19 

19  0% 

94.9% 

SC  16 

14 

13.4% 

1  8  9% 

SCIcum 

12 

12.9% 

18  6% 

C P 1 6  \  SPIcum 

10 

12  9% 

17.1% 

Composite 

2 

1 1 .2% 

14  5% 

Rax leigh  (  I -in  Reg) 

21 

206  6% 

1225  8% 

Rax  leigh  (1-0  Reg) 

449  9% 

5870  6% 

Rax  leigh  MM  At 

N/A 

0  0% 

0  0% 

Beta 

25 

63284  5% 

2096489  2% 

|  TV  Results  for  0%  to  25%  Complete 

Score 

MAPF 

SDAPE 

CPI  3 

14 

24.0% 

30.4% 

C  Pi6 

5 

21  7% 

19.7% 

CPicum 

s 

21.9% 

19  2% 

MM3 

10 

27  4% 

37  0% 

MM 

8 

23  5% 

22  7% 

6  Pic  urn 

1 0 

23  5% 

26  2% 

SC  13 

23 

40  6% 

183.4% 

SCI  6 

16 

26.6% 

28  4% 

SCIcum 

14 

25.6% 

27  4% 

CPlo  \  SPIcum 

1  1 

25  4% 

24.8% 

Composite 

D 

20  7% 

18  6% 

Rax  leigh  1 1  in  Reg  1 

n  n 

107.1% 

55  9% 

Rj>  leigh  iT-G  Reg ) 

24 

1  1 2  0% 

1 13.3% 

Raxieieh  MM  Ah 

N/A 

0.0% 

0  0% 

Beta 

27 

248)45  4% 

4156601.3% 

|  TV  Results  for  >50%  to  75%  Complete 

Score 

MAPF 

SDAPF 

CPI  3 

s 

9.3% 

9.0% 

CPIo 

3 

8.5% 

9  2% 

CPicum 

4 

8.5% 

9.4% 

SIM  3 

16 

1 1 .2% 

10  9% 

SPio 

♦  "I 

9.9% 

10  0% 

SPIcum 

7 

9.0% 

9.5% 

SC  13 

18 

12  1% 

:  1  9% 

SC  1 6 

14 

10  !% 

1 0  4% 

SCIcum 

7 

8  9% 

9  6% 

CPIo  \  SPIcum 

10 

9.3% 

9.8% 

Composite 

4 

8.5% 

9  4% 

Rav leigh  d  m  Reg.) 

23 

303  8% 

1421  1% 

Ra\  leigh  (T-G  Reg) 

7 

366  9% 

1110  1% 

Rax  leigh  MM  AF 

N/A 

0  0% 

0.0% 

Beta 

2i) 

48  2% 

83  2% 

Best  Performing  Method(s) 


Overall 
0%  -25% 
>25%  -  50% 
>50%  -  75% 
>75%  -  ]  00% 


Composite 

Composite 

Composite 

CP16 

CPicum  Composite 


[  TY  Results  for  >25%  to  50%  Complete  * 

Score 

MAPF 

SDAPF 

CP13 

5 

14.9% 

1  7  4% 

CPI6 

5 

14.4% 

17.5% 

CPicum 

6 

14.3% 

17.9% 

SP13 

17 

1  8  7% 

1 8.9% 

SP16 

15 

17.7% 

18  9% 

SPIcum 

1  ! 

16.2% 

1 8  0% 

SC  1 3 

20 

20  4% 

21  4% 

SC  16 

17 

18.0% 

1  9  2% 

SCIcum 

12 

15  3% 

18  5% 

CPI6  x  SPIcum 

12 

16  2% 

18.2% 

Composite 

4 

14  2% 

17.8% 

Rayleigh  (Lin  Reg) 

22 

200  4% 

519  6% 

Rayleigh  (T-G  Reg) 

24 

309  1% 

1155.5% 

Rayleigh  MMAF 

N/A 

00% 

00% 

Beta 

26 

882.9% 

4851  4% 

j  TY  Results  for  >75%  to100%  Complete  | 

Score 

MAPF 

SDAPF 

CPI  3 

15 

5  6% 

5.0% 

CP16 

1 1 

5.2% 

4.5% 

CPicum 

3 

4.7% 

3.5% 

SPI3 

9 

4.9% 

4  4% 

SP16 

5 

4  5% 

3  9% 

SPIcum 

6 

4.8% 

3.8% 

SC  1 3 

17 

5  9% 

7  4% 

SC16 

11 

4  9% 

4.7% 

SCIcum 

6 

4  9% 

3  7% 

CPI6  x  SPIcum 

14 

5.6% 

4  9% 

Composite 

3 

4.7% 

3  5% 

Rayleigh  (Lin  Reg) 

21 

212.2% 

1657.2% 

Ravleigh  (T-G  Reg) 

23 

695  7% 

9132.7% 

Rayleigh  MMAF 

N/A 

0  0% 

0  0% 

Beta 

19 

1 2  8% 

52  4% 
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CY95  Results  for  Overall  Performance 


Score 

MAPH 

SHAPE 

CPI  3 

10 

1 3  0°/o 

19  3% 

CP!6 

5 

1 1  7% 

14  4% 

CPIcum 

4 

1  1  6% 

14.4% 

SP13 

1  6 

1 5  2% 

24  2% 

SPlo 

7 

13  0% 

1 7.0% 

SPIcum 

9 

13.0% 

19  0% 

SC  13 

18 

19  4% 

92.3% 

SCI  6 

13 

14  1% 

19  7% 

SCI  cum 

13 

1 3  6% 

20  0% 

CPio  \  SPIcum 

9 

i  3  6% 

18  0% 

Composite 

i  1  3% 

14.0% 

Ra\leigh  < ! an  Reel 

237.0% 

2986.7% 

RaGcigh  (T-G  Reg) 

23 

300.8% 

2406.3% 

RaGeieh  MMA F. 

20 

235  6% 

494  7% 

Best  Performing  Method(s) 

Overall  Composite 

0%  -  2  5%  C om  posite 

>25%  -  50°/o  CPIcum 

>50% -75%  CP16 

>75%-  100%  SPI6 


2399586.0%! 


CY 95  Results  for  0%  to  25%  Complete 


eh  (  1  -G  Reg) 
eh  M  MAP 


CY95  Results  for  >25%  to  50%  Complete 


Score!  MAPP  SDAPE 


CPI6 

5 

13.6% 

1 7  7% 

CPieum 

4 

13.3% 

18  1% 

SPI3 

17 

18  1% 

19.4% 

SP16 

15 

17.1% 

19  4% 

SPIcum 

10 

1 5  5% 

1 8  5% 

SC  13 

20 

20.0% 

22.0% 

SC16 

17 

1 7  7% 

19  7% 

SCI  cum 

1  1 

15  1% 

CP16  n  SPIcum 

12 

16.1% 

18.6% 

Composite 

5 

13.4% 

18  1% 

Rayleigh  (Lm  Reel 

26 

612  9% 

64269% 

Rayleigh  CI-G  Reg) 

203  4% 

501.0% 

Rayleigh  MMAE 

24 

353.5% 

615.7% 

CY95  Results  for  >50%  to  75%  Complete 


Score  MAPF  SDAPE 


SP13 

16 

1 1 .2% 

10.4% 

S  P 1 6 

14 

10.0% 

9.9% 

CY95  Results  for  >75%  to!00%  Complete 


CPI  3 

13 

7.6% 

7  8% 

CP16 

12 

7  6% 

7.5% 

CPIcum 

6 

6.8% 

6.0% 

SP13 

5 

6  7% 

6.0% 

SC  leum _ 

CPI6  x  SPIcum 


Beta 


317  4% 
82.0% 


Beta 


17 
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|  TV  Results  for  AH  Aircraft  Data  | 

Score 

MAPE 

SDAPE 

CPI  3 

10 

15  1% 

18.8% 

CP16 

3 

13.7% 

18.1% 

CPIcum 

13 

15  1% 

19  5% 

SPi3 

17 

15.8% 

19.8% 

SPlO 

6 

14  1% 

18  7% 

S  Pi  cum 

14 

15  4% 

19  4% 

SC  1 3 

16 

16  2% 

19  4% 

SCI6 

6 

14.2% 

18  4% 

SClcurn 

12 

15  3% 

19.2% 

CPU'  \  SPIcum 

13  7% 

1 7  8% 

Composite 

1  1 

15  0% 

19  4% 

RaGeigh  i  i  in  Ret: ) 

20 

104  6% 

292.7% 

Ravleigh  iT-G  Regi 

22 

2138  5% 

18310.9% 

Ravleigh  VMM 

N  A 

0.0% 

0  0% 

Beta 

24 

627763.6% 

6700589.2% 

TV  Results  for  Aircraft  Data  0%  to  25% 
Complete 

Score 

MAPE 

SHAPE 

CP13 

10 

29  0% 

22.9% 

CP16 

13 

29  9% 

22.7% 

CPIcum 

14 

29  8% 

23.0% 

SP13 

15 

30.9% 

22  9% 

SPlo 

4 

28.5% 

21  5% 

SPIcum 

9 

29.2% 

22  0% 

SC  13 

13 

31.2% 

21  7% 

so 

5 

27  5% 

21.7% 

SCicum 

7 

29  2% 

21.6% 

CPI6  \  SPIcum 

-* 

27  5°o 

21  4% 

Composite 

13 

29  3% 

23.0% 

Ravictgh  i  Lin  Reg) 

:n 

87  7% 

23  3% 

Ravleigh  ('i'-G  Reg) 

20 

86  6% 

28.0% 

Ravleigh  MMAE 

\  A 

0  0°  o 

0.0% 

Beu 

23 

2241748.2% 

12649935  8% 

TV  Results  for  Aircraft  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

1  1 

7.5% 

2  8% 

CPU1 

1  1 

7  0% 

3.1% 

CPIcum 

6 

6  4% 

2.9% 

SP13 

to 

7.2% 

2.9% 

SP1* 

9 

6  8% 

3.0% 

SPIcum 

8 

6.3% 

3.2% 

SC  13 

12 

8  0°  o 

2.8% 

SC  16 

9 

7  4% 

2.6% 

SCicum 

5 

6.0% 

3  0% 

CP16  x  SPIcum 

10 

6.7% 

3.2% 

Composite 

6 

6.3% 

3.0% 

Rayleigh  i  Lin  Reg.i 

20 

228.0°  o 

613  7% 

Ra>  ietgh  f  T-G  Reg  i 

*>*> 

467  5% 

806.5% 

Ravi  etch  M  MAE 

N  A 

0.0% 

0.0% 

Beta 

18 

J  T7  ">o  0 

190.8% 

Best  Performing  Method(s) 

Overall  CP16  x  SPIcum 

0%  -  25%  CP16  x  SPIcum 

>25% -50%  CPI3  CP16 


>50%  -  75%  SCI  cum 

>75%- 100%  CPIcum  SP16  Composite 


TV  Results  for  Aircraft  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SHAPE 

CPI3 

4 

23.4% 

22  5% 

CPI6 

4 

23  2% 

22.9% 

CPIcum 

5 

23.0% 

23.5% 

SPI3 

13 

24  7% 

24  8% 

SPI6 

18 

26  8% 

24.9% 

SPIcum 

14 

25.4% 

24.2% 

SC  13 

1  1 

25  1% 

24  1% 

SC16 

17 

27.0% 

24  4% 

SClcurn 

11 

^5  n% 

23.9% 

CPI6  x  SPIcum 

!  1 

25.4% 

23.5% 

Composite 

6 

23  4% 

23  4% 

Ravleigh  (Lin  Reg) 

21 

62  4% 

47.7% 

Ravleigh  (T-G  Reg) 

25 

420  7°  o 

131 1.5% 

Ravleigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

23 

227  0% 

358  6% 

TV  Results  for  Aircraft  Data  >75%  to  100% 
Complete 

||  Score 

MAPE 

SDAPE 

CP13  ||  9 

4.6% 

3.6% 

CP16 

7 

4  4% 

3  4°  o 

CPIcum 

3 

4  2°  o 

3.2% 

SPI3 

10 

4  6% 

3.8% 

SPI6 

3 

4. !  % 

3  3% 

SPIcum 

4 

4.2% 

3.3% 

SCI3 

12 

5  0% 

4.2% 

SC  16 

7 

4  4% 

3.4% 

SCicum 

6 

4.3% 

3  4% 

CP16  x  SPIcum 

9 

4.6% 

3  6% 

Composite 

3 

4.2% 

3.2% 

Rayleigh  (Lin  Reg) 

16 

92  7% 

276  2% 

Rayleigh  (T-G  Reg) 

18 

4252.3% 

26931.3% 

Ravleigh  MMAE 

N  A 

0.0% 

0  0% 

Beta 

14 

39  3% 

145.1% 
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CY95  Results  for  All  Aircraft  Data  | 

Best  Performing  Method(s) 

Overall  SP16 

0%  -  25%  CPI6  x  SPIcum 

>25% -50%  CPI3  CPI6  CPIcum 

>50%>  -  75%  CPIcum  SCIcum  Composite 

>75% -100%  SPI6 

Score 

MAPH 

SDAPE 

CPI  3 

6 

16  7% 

1 7  7% 

CPU- 

8 

1 6.3°o 

1  8  0% 

CPI  cum 

12 

17  0% 

18  1% 

SP13 

14 

1 7  2°  o 

18.8% 

SPit* 

"> 

16.1% 

17  6% 

SPIcum 

12 

1 7.3% 

1 7  9% 

SC  13 

17 

17  7% 

19  0% 

St.' IS 

3 

16.2% 

17  6% 

SCIcum 

10 

1 7.2% 

17  8% 

CPIS  \  SPIcum 

16.3% 

17  0% 

Composite 

]0 

16  0% 

18  0% 

Ravlemh  i  1  in  Reg ) 

10 

83  7% 

146  3% 

Ravi  cm!)  H-G  Ree  ; 

21 

164  8% 

251  1% 

Ra\  Icmh  MM  A  K 

23 

350  2% 

598.9% 

Beta 

->s 

713681  7% 

7670335.1% 

CY95  Results  for  Aircraft  Data  0%  to  25% 
Complete 

CY95  Results  for  Aircr 
Com 

aft  Data  >25 
plete 

%  to  50% 

Score 

MAPf 

SDAPE 

Score 

MAPE 

SDAPE 

CPI  3 

8 

27.5% 

20.4% 

CP13 

4 

22.5% 

23  4% 

CPIs 

v 

2o  5% 

210% 

CPI6 

4 

22  3% 

23  7% 

Oleum 

12 

26.0%, 

22.2% 

CPIcum 

4 

22  0% 

24  0% 

SP13 

13 

20  0% 

22  1% 

SP13 

10 

24  0% 

24.7% 

SPlo 

s 

26  1% 

20.5% 

SPI6 

17 

26  1% 

25.0% 

SPIcum 

8 

26  0% 

20.5% 

SPIcum 

13 

24  5% 

24  6% 

SC!  3 

12 

20  7% 

20.8% 

SC13 

12 

24.2% 

24  7% 

SC  I  ts 

s 

24.8% 

20  8% 

SC  16 

17 

26.0% 

25  1% 

SCI  turn 

7 

26.2% 

20  8% 

SCIcum 

10 

24  1% 

24  6% 

CPie  \  SPIcum 

2 

24.5% 

20  4% 

CPI 6  x  SPIcum 

12 

24.3% 

24  6% 

^  ontposiie 

1 1  • 

26  5% 

22  1% 

Composite 

6 

22.3% 

24  1% 

Racteiah  i  Lin  Ree  i 

18 

87.4% 

25  4% 

Ravleigh  (Lin  Reg) 

20 

63.6% 

51.7% 

Ravleigh  i  T-G  Ree  : 

18 

86  1% 

30.9% 

Ravleigh  (T-G  Reg) 

23 

205.5% 

418  1% 

Ra\  lei-'ii  M MAP 

21 

506  6% 

786  0% 

Ravleigh  MMAE 

26 

540.9% 

691  7% 

Beta 

23 

2548471  1% 

14483026  6% 

Beta 

23 

264. 3% 

370  3% 

CY95  Results  for  Aircraft  Data  >50%  to  75% 
Complete 

CY95  Results  for  Aircraft  Data  >75 
Complete 

%  to  100% 

1  Score  |  MAPLj  SI)  A  PL 

Score 

MAPE 

SDAPE 

CPI3 

13 

7  7% 

3  4% 

CPI3 

0 

100% 

9.3% 

CP  IP 

Hi 

7  7% 

3.0% 

CP16 

16 

11.8% 

12.5% 

CPI  cum 

5 

7  3% 

2  6% 

CPIcum 

10 

1 1  0% 

8.5% 

SP13 

]  2 

7.4% 

3.4% 

SP13 

5 

9.5% 

8.6% 

SPI6 

0 

7.2% 

3  3% 

SPI6 

3 

9  8% 

8.1% 

SPIcum 

0 

6.8% 

3.2% 

SPIcum 

8 

10.8% 

8  3% 

SC  13 

14 

8  2% 

3.3% 

SCI3 

0 

9.9% 

112% 

V  ]f 

14 

7.8% 

3  4% 

SCI6 

10 

10  5% 

9  3% 

St  ’ ! c  urn 

5 

6  0% 

2.8% 

SCIcum 

1  1 

11.1  % 

8  5% 

CPio  \  SPIcum 

1  ! 

7  4% 

3.3% 

CP16  x  SPIcum 

17 

1 1  9% 

12  5% 

Com  pi  is:  te 

5 

7  2% 

2.7% 

Composite 

10 

1 1  0% 

8  5% 

Ravleigh  1 1  in  Ret: } 

17 

1  16  3°  c 

1 70  7% 

Ravleigh  (Lin  Reg) 

10 

80.5% 

198  8% 

Ravi e tab  iT-G  Rect 

20 

191.1% 

176  6% 

Ravleigh  (T-G  Reg) 

23 

166  8% 

215  4% 

Rao  iei eh  MMAF 

23 

318  0° 

570  0% 

Ravleigh  MMAE 

23 

148  6% 

338  8% 

Beta 

20 

]  -vt  70 

188  0% 

Beta 

23 

1  1  1.3% 

396  7% 
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|  TV  Results  for  All  Armored  Vehicle  Data 

■ 

Best  Performing  Method(s) 

Overall  CPI6  x  SPlcum 

0%  -  25%  SP16 

>25%  -  50%  CP13  CP16  x  SPlcum 

>50%  -  75%  CPI3 

>75%-  100%  SPI3 

Score 

MAPI 

SDAPI 

CP!  3 

15 

1  7  49 

18  6° 

CP16 

12 

1 6.0% 

18.1% 

CPI  cum 

20 

1 9.6% 

19.6% 

SPI3 

13 

1 7  7% 

16  4% 

SPIb 

10 

1 6  5°  c 

15  3% 

SPlcum 

12 

17  8% 

14  9% 

SC  1 3 

16 

18  7% 

17  0% 

SC  lb 

4 

14.2% 

13  0% 

SCicum 

6 

14.8% 

14.4% 

CP  16  \  SPlcum 

n 

12.6% 

1 1  3% 

Composite 

ip 

1 9  6% 

18  6% 

Ra\  ieigh  1 1 .in  Reg) 

106  1% 

120.0% 

Ra\ ieigh  . T-G  Reg) 

24 

157.8% 

209  0% 

Ra  sleigh  MM  At 

N  A 

0  0% 

0.0% 

Beta 

26 

325  8% 

926.3% 

TV  Results  for  Armored  Vehicle  Data  0%  to  25% 

1  TY  Results  for  Armored  V  ehicle  Data  >25%  to  1 

Com 

plete 

50%  Complete 

Score 

MAPF. 

SDAPF 

Score 

MAPF 

SDAPF 

CP  13 

18 

40.6% 

18  0% 

CPI  3 

5 

17.4% 

6.5% 

CPI  6  ! 

op 

45  1% 

16.7% 

CP16 

6 

16  7% 

8  8% 

Cl>[cum 

19 

44.1% 

16.7% 

CPIcum 

10 

22.0% 

9.1% 

SPI3 

1  1 

26  5% 

14  9% 

SPI3 

20 

32.0% 

21.1% 

SPib 

4 

25.6% 

103% 

SP16 

2"> 

32.3% 

21.5% 

SPlcum 

1  1 

33  0% 

14  3% 

SPlcum 

16 

24.3% 

10.1% 

SC  1 3 

X 

30.6% 

14.0% 

SCI  3 

16 

23.7% 

16.9% 

SC  16 

"7 

22  1% 

14  6% 

SC  16 

15 

22  6% 

18  1% 

SCicum 

]  j 

31  7% 

14  4% 

SCicum 

7 

i  4  7% 

9  4°  o 

CPI6  \  SPlcum 

8 

25  4% 

14  6% 

CPI6  x  SPlcum 

5 

14  4% 

9.1% 

Composite 

i  7 

42  5% 

16  2% 

Composite 

10 

22  8% 

8.0% 

Rayleigh  (tin  Reg) 

24 

88.5% 

312% 

Ravieigh  ( Lin  Reg) 

25 

118  4% 

108.3% 

Ra>  ieigh  1 T -G  Regi 

14 

73  0% 

13.2% 

Ravieigh  (T-G  Ree) 

j  27 

313.3% 

368.2% 

Ra>  ieigh  MMAI: 

N  A 

0.0% 

0  0% 

Ravieigh  MMAF 

N  A 

0.0% 

0.0% 

Beta 

26 

1 1  17.9% 

1542.0% 

Beta 

16 

20.0% 

25.5% 

1  TV  Results  for  Armored  Vehicle  Data  >50%  to  1 

TV  Results  for  Armored  V  ehicle  Data  >75%  to  1 

7 

5%  Complete 

100%  Complete 

Score 

MAPF-I 

SDAPF 

l 

Score 

MAPF 

SDAPF 

CPI  3  | 

7 

6  1% 

4  2% 

CPI3 

8 

3  5% 

3  1% 

CPI6 

10 

7.1% 

4  8% 

CP16 

1  1 

3  7% 

3.5% 

CPIcum 

12 

8.3% 

5.3% 

CPIcum 

3 

3  1% 

2  9% 

SP13 

9 

10  4% 

3.5% 

SP13 

2 

3  1% 

2.7% 

SPIb  ! 

10 

1 1.7% 

3  4% 

SP16 

8 

3.7% 

3.0% 

SPicum 

i  1 

1  1  5% 

3  7% 

SPlcum 

6 

3.4% 

3  0% 

sen 

23 

15.5% 

19  6% 

SCI3 

11 

3.8% 

3.3% 

SC  16 

21 

13.0% 

7.3% 

SCI6 

12 

4.3% 

3  3% 

SCicum  ■ 

9 

8.8% 

3.8% 

SCicum 

4 

3.1% 

3.0% 

CPIbx  SPlcum 

18 

1 1  5% 

7.2% 

CPI6  x  SPlcum 

12 

3  7% 

3  6% 

Composite  ! 

1  ! 

9.2% 

4.1% 

Composite 

4 

3.2% 

2  9°  o 

Rayleigh  1 1  in  Reg) 

27 

93  9% 

120  7% 

Ravieigh  (Lm  Regi 

17 

1 19.7% 

178  4% 

Rayleigh  (T-G  Reg)  j 

7  s 

75.2% 

94.1% 

Ravieigh  (T-G  Reg) 

17 

151.3% 

148.9% 

Ravieigh  MMAI  j 

N  A 

0.0% 

0  0% 

Ravieigh  MMAF 

N  A 

0.0% 

0.0% 

Beta 

i  1 

6  8% 

6  0% 

Beta 

8 

3  5% 

3.1% 
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j  CY95  Results  for  All  Armored  Vehicle  Data 

Best  Performing  Method(s) 

Overall  CP16  x  SPIcum 

0%  -  25%  SP16 

>75%  -  50%  CPI3  CPI6  SC  Icum 

>50%  -  75%  CP13 

>75%-  100%  SC13 

CPI6  x  SPIcum 

Score 

MAPE 

SDAPE 

cpg 

16 

16  2% 

1 5  6% 

CPlo 

|  14 

15  2°o 

15  7% 

CPI  cum 

20 

19  1% 

16  5% 

SP13 

13 

1 5  6°  o 

14  9% 

SPlo 

9 

1  5  0% 

13  7% 

SPIcum 

12 

16.2% 

12.1% 

SC  13 

13 

1  5  8% 

14  6% 

SC  IP 

4 

11.0% 

1 1 .0% 

SCicuni 

6 

13.1% 

113% 

CPiP  \  S Pic  urn 

10  7% 

8  1% 

Composite 

17 

18  8% 

15  6% 

Rayleigh  ( l.in  Reg) 

22 

1 50  0°  o 

225  6% 

Ravleigh  (T-G  Reg  i 

26 

585  2% 

1 059  3% 

Ravleigh  MMAE 

28 

588.1% 

1217.9% 

Beta 

24 

180.9% 

380  4% 

I  CY95  Results  for  Armored  V  ehicle  Data  0%  to 

|CY95  Results  for  Armored  Vehicle  Data  >25%  to 

25%  Complete 

[  50%  Complete 

Score 

MAPE 

SDAPE 

Score 

MAPE 

SDAPE 

CPI  3 

18 

35  6% 

16  0% 

CPI3 

5 

15.5% 

3  5% 

CPIt> 

18 

40  5% 

15.1% 

CP16 

5 

14.9% 

5.8% 

C  Pi  cum 

19 

39  2% 

15  5% 

CPlcum 

i  1 1 

19  3% 

9.4% 

SPli 

12 

194% 

15.2% 

SP13 

■S') 

28  8% 

22  0% 

SPR 

3 

18  7% 

8  1% 

SP16 

24 

29  2% 

22.5% 

SPicum 

12 

26.9% 

13.0% 

SPIcum 

i  j5 

20  9% 

10  8% 

SC.  13 

16 

24.4% 

16.6% 

SC  13 

18 

2 1 .8% 

16.0% 

Sc':- 

4 

15.9% 

9.8% 

SC  16 

15 

19.5% 

18.5% 

SC  i  cum 

10 

25  8% 

12  1% 

SCIcum 

5 

12  5% 

7  2% 

0*16  x  SPIcum 

6 

19  3% 

9  8% 

CP16  x  SPIcum 

5 

14.2% 

7.0% 

Composite 

15 

37  4% 

14  8% 

Composite 

12 

19.9% 

86% 

Ra\  leigh  ( l.m  Reg.  i 

25 

90  2% 

28  7% 

Ravleigh  <.Lin  Reg) 

26 

139  6% 

1 25  9°  o 

Rax  lei  eh  (  1  -G  Reg) 

14 

73.5% 

8  4% 

Rayleigh  (T-G  Reg) 

30 

1003.8% 

1430.8% 

Ra\  leigh  MM  At 

29 

1474.0% 

2519  7% 

Ravleigh  MMAE 

28 

828  2% 

843. 2% 

Beta 

610.9% 

5 1 7.5% 

Beta 

19 

21.3% 

19  4% 

|CY95  Results  for  Armored  Vehicle  Data  >50%  to 

CY95  Results  for  Armored  Vehicle  Data  >75%  tol 

|  75%  Complete 

100%  Complete 

Score 

MAPF. 

SDAPE 

Score 

MAPE 

SDAPE 

CP!  3 

5 

7  8°  o 

34% 

CP13 

12 

4.2% 

3.1% 

CP* 

8 

7  8% 

4  9% 

CP16 

12 

4.7% 

2.9% 

CPlcum 

9 

1 1  3% 

3.4% 

CPlcum 

13 

5.1% 

2.8% 

SP13 

0 

1 2  6% 

2.5% 

SP13 

5 

3.6% 

2.5% 

SPR 

14 

1 3  7% 

4  1% 

SPJ6 

12 

4.2% 

3.1% 

SPIcum 

14 

1 3  4% 

4.5% 

SPIcum 

10 

4.6% 

2.8% 

SC  13 

20 

13  7% 

10  6% 

SCI3 

2 

2.8% 

2.3% 

SCR 

13 

8  8% 

5  3% 

SCI6 

10 

3  9% 

3.0% 

Sc  icum 

12 

8  0% 

5  5% 

SCIcum 

14 

5.1% 

2.9% 

CPlt'  \  SPIcum 

1  i 

8.2% 

5.1% 

CPI6  x  SPIcum 

13 

4.7% 

3.0% 

Composite 

9 

12  0% 

3  1% 

Composite 

1  1 

5  0% 

2.7% 

Ra>  lei  eh  ( I.  in  Reg  ! 

25 

1 72  7% 

229.4% 

Ravleigh  ( Lin  Res ) 

20 

1 77  9% 

356.0% 

Ravleigh  tl-G  Reg ) 

22 

1 04  8% 

134.2% 

Ravleigh  (T-G  Reg) 

-jo 

876  9% 

1330  4% 

Ruv  ieigh  MM  At 

2s 

269.7% 

156.3% 

Ravleigh  MMAE 

18 

102.9% 

162.7% 

Beia 

g 

6  1% 

5.3% 

Beta 

6 

3  1% 

2.8% 
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|  TV  Results  for  AM  Ballistic  Missile  Data  | 

Score 

MAPH 

SDAPE 

CP!  3 

18 

1 4  4% 

33  0% 

CPI  6 

6 

1 2  2% 

19.7% 

CPI  cum 

4 

1  1.6% 

19.1% 

SPI3 

20 

15.7% 

36.9% 

SPIo 

12 

12.9% 

22.5% 

S Pie urn 

8 

12.6% 

21.0% 

SC  13 

"> 

27.9% 

209  0% 

SClo 

16 

14.0% 

27  8% 

SCIcum 

10 

12.7% 

214% 

C  P  16  x  SPicum 

14 

13.9% 

23  4% 

Composite 

*> 

1  1.3% 

19.0% 

Ras  leigh  t  i  in  Reg  i 

24 

184.8% 

578.5% 

Rayleigh  ^  T-G  Reg) 

26 

314  2% 

1477.4% 

Rayleigh  MM  At' 

N  A 

0  0% 

0  0% 

Beta 

28 

1294  7% 

8799  6% 

TY  Results  for  Ballistic  Missile  Data  0%  to  25% 
Complete 

Score 

MAPH 

SDAPE 

CPI  3 

17 

28  9% 

60  8% 

C  Pr6 

6 

22  0% 

27  3% 

l  Pie  uni 

4 

20  2% 

24  9% 

SIC  3 

21 

33  0% 

68  8% 

SPlo 

13 

25.8% 

34  4% 

SPicum 

9 

22  9% 

29  8% 

SC  13 

24 

87.0% 

437  9% 

SCio 

18 

31.8% 

49  1% 

SCR  urn 

1  ! 

24.4% 

30  7% 

CPIo  \  SPicum 

16 

29.0° ,» 

36  6% 

Composite 

2 

18  9% 

24  8% 

Ra\ leigh  <  Lin  Reg) 

2s 

1  16  7% 

105  4% 

Ka>  ieiuti  «  !  -Ci  Reg  ! 

16 

97  5% 

28.3% 

Ka\ IciL’ii  V1MAF. 

N  A 

0.0% 

0.0% 

Beta 

2K 

5016.1% 

18459  3% 

|  TV  Results  for  Ballistic  Missile  Data  >50%  to 
|  75%  Complete 

Score 

MAPH 

SDAPE 

CP!  3 

4 

10.5% 

12.5% 

CPIo 

7 

10.6% 

13.5% 

CPIcum 

u 

10  4% 

14.0% 

SP13 

14 

11  1% 

14.1% 

SP16 

8 

10  3% 

14  0% 

SPicum 

Hi 

1 0  6% 

13.9% 

SC  13 

8 

1  1.1% 

13  2% 

SCI  6 

7 

10  4% 

1 3  8% 

SCIcum 

]  2 

10.5% 

14  2% 

CPIo  x  SPicum 

9 

10  9% 

13  7% 

Composite 

8 

10.4% 

13  9% 

Rayleigh  1 1  in  Reg) 

1  0 

96  2% 

136  8% 

Ru\  teieh  PI  -G  Reg  i 

21 

381  2°  o 

1486.4% 

Rayleigh  MMAE 

N  A 

0  0% 

0  0% 

i  A'  U: 

J 

49  4% 

99  3% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%  -  100% 


Composite 
CPI6  SCI6 
CPI  3 
SC  13 


TY  Results  for  Ballistic  Missile  Data  >25%  to 
50%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

7 

15.2% 

26  1% 

CPI6 

6 

14.2% 

26.6% 

CPIcum 

9 

14.2% 

27  5% 

SPI3 

9 

17  0% 

25  7% 

SPI6 

1  1 

15.3% 

26  8% 

SPicum 

15 

15.5% 

28.0% 

SC  13 

16 

17  8% 

27.5% 

SC  16 

6 

14  4% 

26  2% 

SCIcum 

13 

14  7°  6 

28.1% 

CP16  x  SPicum 

10 

14.7% 

27.1% 

Composite 

8 

14  0% 

27.5% 

Rayleigh  (Lin  Reg) 

22 

269.5% 

764.2% 

Rayleigh  (T-G  Reg) 

20 

163  9% 

335.1% 

Rayleigh  MMAE 

N  A 

0.0% 

o 

o 

Bela 

24 

1018  6% 

3432. 8%| 

TY  Results  for  Ballistic  Missile  Data  >75%  to 
100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

12 

6  1% 

4.2% 

CPI6 

14 

6.2% 

4.3% 

CPIcum 

8 

5  8% 

4.0% 

SPI3 

9 

5.6% 

4.2% 

SP16 

9 

5.4% 

4  4% 

SPicum 

14 

6.0% 

4.5% 

SC  1 3 

n 

5  3% 

3.8% 

SC  16 

5 

5  5% 

3  9% 

SCIcum 

1  1 

6  0°  o 

4.2% 

CP16  x  SPicum 

17 

6.5% 

4  5% 

Composite 

9 

5  8% 

4  1% 

Rayleigh  (Lin  Reg) 

21 

219.3% 

732.1% 

Ravleigh  (T-G  Reg) 

23 

441.7% 

2089  4% 

Rayleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

19 

10  7% 

21.9% 
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C\95  Results  for  All  Ballistic  Missile  Data  ] 

Score 

MAPE 

SDAPE 

CPI  3 

17 

13.5% 

33  4% 

CPI6 

6 

11.3% 

20  6% 

C  Pic  urn 

4 

10  6% 

19  9% 

SP13 

19 

1  5  4% 

37.8% 

SPIo 

12 

12  5% 

24  0% 

S  PI  cum 

8 

12.1% 

22  6% 

SC  13 

:i 

27.1% 

202  4% 

SC  16 

10 

13  5% 

28  7% 

SC  i  cum 

10 

12  2% 

22  9% 

CPI6  \  SPicum 

14 

13.3% 

24  8% 

Composite 

2 

10.4% 

19.8% 

Ra\  Ieigh  ( Lin  Reg  i 

25 

183  9% 

640.7% 

Ravleigh  (T-G  Reg) 

25 

198.5% 

523.1% 

Ra>  ieigh  MMAL 

2> 

214  1% 

418.9% 

Bela 

29 

2855.8% 

33005  4% 

CY95  Results  for  Ballistic  Missile  Data  0%  to 
25%  Complete 

Score 

MAPE 

SDAPE 

CP  13 

37 

28  1% 

61.1% 

CPI  6 

6 

21  2% 

28  4% 

CPicum 

4 

18.7% 

26  2% 

5  PI  3 

2! 

33.8% 

70  0% 

SPio 

13 

26.3% 

37  4% 

SPicum 

9 

23  9% 

32  8% 

SC  13 

24 

86  ]  % 

423.5% 

SC  in 

18 

32  1% 

50.3% 

SC!  cum 

1  1 

25  3% 

33  1% 

CPio  \  SPicum 

16 

29.6% 

oo 

oc 

o 

Composite 

2 

17.9% 

26  1% 

Ravleigh  (Lin  Reg) 

25 

115  0% 

85  6% 

Ravleigh  Cl  -G  Reg) 

16 

99  0% 

29  9% 

Ra>  Ieigh  MMAL 

28 

510.8% 

708  5% 

Bela 

30 

12362.9% 

71375.1% 

j  CY95  Results  for  Ballistic  Missile  Data  >50%  to 
|  75%  C  omplete 

Score 

MAPE 

SDAPF. 

CPI  3 

1 

10.3% 

12.9% 

CPio 

6 

1 0  6% 

13  8% 

CPI  cum 

10 

10  7% 

14.1% 

SPI3 

14 

11  1 0  0 

1 4  2% 

SPI6 

8 

10.6% 

14.0% 

SPicum 

9 

30.9% 

13.8% 

SC  13 

7 

10.8% 

13.4% 

SC  16 

8 

10.5% 

14  1% 

SC  Sc  urn 

1  1 

1 0.6% 

14  3% 

CP  16  x  SPicum 

9 

10.8% 

13  9% 

Composite 

9 

10  7% 

14  0% 

Ra>leigh  i  Lin  Reg) 

18 

76  4% 

95  0% 

Ra\ Ieigh  H-G  Reg) 

23 

327.4% 

890  8% 

Ravleigh  MMAE 

21 

131  8% 

3  77.4% 

Beta 

_ L2 _ i 

50  3% 

99  9% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


Composite 
CPI3  CP16  SCI6 
CPI  3 
SCI3 


CY95  Results  for  Ballistic  Missile  Data  >25%  to 
50%  Complete 

Score 

MAPE 

SDAPE 

CP13 

6 

15  1% 

27.3% 

CP16 

6 

14.2% 

27.9% 

CPicum 

10 

14  2% 

28  8% 

SP13 

8 

1 7.2% 

27.0% 

SP16 

10 

15  5% 

28.3% 

SPicum 

15 

15  8% 

29.5% 

SC  1 3 

15 

17.9% 

28.5% 

SC  16 

6 

14  8% 

27  5% 

SCI  cum 

13 

15  1% 

29  5% 

CPJ6  x  SPicum 

10 

15.1% 

28  4% 

Composite 

9 

14  1% 

28  8% 

Ravleigh  (Lin  Reg.) 

so 

308.0% 

1079  3% 

Rayleigh  (T-G  Reg i 

19 

126  3°  o 

143.7% 

Rayleigh  MMAE 

2° 

33 1.5% 

526.2% 

Beta 

25 

1050.4% 

3560.0% 

CY95  Results  for  Ballistic  Missile  Data  >75%  to 
100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

8 

4.0% 

3.5% 

CPI6 

8 

4.0% 

3.5% 

CPicum 

4 

3.8% 

3.2% 

SP13 

7 

3.9% 

3  5% 

SPI6 

8 

3.6% 

3  8% 

SPicum 

7 

3.9% 

3.5% 

SCI3 

3 

3.5% 

3.3% 

SC  1 6 

6 

3.6% 

3.6% 

SCIcum 

6 

3  9% 

3.3% 

CPI6  x  SPicum 

1  ! 

4.2% 

3  7% 

Composite 

5 

3.8% 

3.3% 

Rayleigh  (Lin  Reg) 

20 

201.9% 

564  3% 

Ravleigh  (T-G  Reg) 

18 

187.2% 

388.5% 

Ravleigh  MMAE 

1 6 

76.2% 

108.6% 

Beta 

14 

10.5% 

217% 
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TV  Results  for  All  Electronic  Warfare  Data 

Score 

MAPE 

SDAPE 

CPi3 

7 

13.1% 

1  5.2% 

CPI  6 

4 

12  8% 

15  1% 

CPI  cum 

8 

13.0% 

15.5% 

SP13 

14 

13  8% 

16.5% 

SP16 

7 

12.8% 

15.5% 

SPIcum 

2 

12.6% 

14  3% 

SC  13 

16 

1 5.4% 

17  7% 

SCI  6 

12 

13.6% 

1 6.3% 

SClcum 

9 

1 3.2% 

15.3% 

CPIS  x  SPIcum 

5 

13.0% 

15.1% 

Composite 

4 

12  8% 

15.1% 

Ravleiph  (Lin  Reu) 

19 

237  5% 

1296  3% 

Rayleigh  (T-G  Reg. 

19 

398  4% 

1  154  8% 

Rayleigh  MMAI'i 

N  A 

0.0% 

0.0% 

Beta 

1702  5% 

13846  9% 

TV  Results  for  Electronic  Warfare  Data  0%  to 
25%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

0 

24  6% 

217% 

CP!  6 

1  1 

24  6% 

22  1% 

CPlcum 

12 

25.1% 

218% 

SP13 

13 

24.0% 

24  7% 

SPI6 

8 

22  6% 

23  1% 

SPIcum 

2 

21  6% 

20  6% 

SC  1 3 

:  18 

28  5% 

24  4% 

SCIo 

1  s 

25  0% 

24  2% 

SCicurti 

ID 

24.5% 

22. 1  % 

CP16  \  SPIcum 

10 

24  !  % 

22.2% 

Composite 

5 

23  9% 

21.7% 

Ravleigh  (I. in  Reg) 

^  A 

126  3% 

59.3% 

Ravleigh  (T-G  Reg) 

22 

1 15.6% 

69.5% 

Ravleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

25 

7563.3% 

29266.1% 

TV  Results  for  Electronic  Warfare  Data  >50%  to 
75%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

4 

12  5% 

12.0% 

CP16 

2 

12  3% 

I  1.9% 

CPlcum 

9 

12.5% 

12.6% 

SP13 

i  7 

15.0% 

1 3.4% 

SP16 

1  1 

13.2% 

12.5% 

SPIcum 

10 

13  4% 

12.1% 

SC  13 

19 

15  9% 

15.2% 

SCI6 

14 

1 3.3% 

13.1% 

SClcum 

1  1 

13.4% 

12.3% 

CPI 6  \  SPIcum 

10 

12  9% 

12.5% 

Composite 

8 

1 2.6% 

12  4% 

Rayleigh  (Lin  Reg) 

25 

663  2% 

2752.1% 

Rav  lei  eh  (  I  -G  Reg.  ) 

23 

525.5% 

1390.9% 

Ravleigh  MMAE 

N  A 

0.0% 

o 

o 

Beta 

21 

57.2% 

63.7%] 

Best  Performing  Method(s) 

Overall  SPIcum 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPIcum 

SPIcum 

CP16 

CPlcum  Composite 


TV  Results  for  Electronic  W  arfare  Data  >25%  to 
50%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

9 

19.3% 

15.0% 

CP16 

6 

19.1% 

14.9% 

CPlcum 

8 

19.2% 

15  0% 

SP13 

14 

21  3% 

15  2% 

SP16 

i  15 

21.3% 

15  6% 

SPIcum 

s 

19.8% 

13.8% 

SCI3 

18 

2 1  8°  o 

15.7% 

SC  16 

18 

21.6% 

15.9% 

SClcum 

7 

19.9% 

14  0% 

CP16  x  SPIcum 

9 

20  0% 

14  1% 

Composite 

6 

19  2% 

14.8% 

Rayleigh  (Lin  Reg) 

23 

287  7% 

462.9°  o 

Ravleigh  (T-G  Reg) 

25 

787  5% 

1 908.6% 

Rayleigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

21 

272.6% 

300.9°  o 

TV  Results  for  Electronic  Warfare  Data  >75%  to 
100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

12 

5.1% 

4.4% 

CPI6 

9 

4.9% 

3  6% 

CPlcum 

2 

4.6% 

2  7% 

SPI3 

6 

4  7% 

3  1% 

SP16 

5 

4  7% 

3.0% 

SPIcum 

4 

4.7% 

2.8% 

SC  13 

16 

5  6% 

6.7% 

SC  16 

14 

5.3% 

4  5% 

SClcum 

6 

4.8% 

3.0% 

CPI6  x  SPIcum 

11 

5  1% 

4.0% 

Composite 

n 

4.6% 

2  7% 

Ravleigh  (Lin  Reg) 

20 

514% 

104.4% 

Rayleigh  (T-G  Reg) 

22 

257.7% 

654.2°  o 

Ravleigh  MMAE 

N  A 

0  0°  o 

0.6% 

Beta 

18 

1 1  6°  o 

29  4°  o 
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0  95  Results  for  All  Electronic  Warfare  Data 

Score 

MAPE 

SDAPE 

CP13 

10 

14.0% 

14.1% 

CPJ6 

6 

13.6% 

14  0% 

CPIcum 

8 

13  7% 

14  1% 

SP13 

14 

14  4% 

1  5  9% 

SPlf- 

10 

13.8% 

14  9% 

SPicum 

13  4% 

13.2% 

SC  1 3 

17 

16.1% 

17  1% 

SC  1 6 

15 

14  7% 

15  9% 

SClcum 

1  l 

14  1% 

14  1% 

CP16  \  SPlcum 

1  I 

14  1% 

14. 1  % 

Composite 

4 

13  5% 

13  8% 

Ra\  leigh  tLin  Reg  i 

^  3 

765.8% 

7858  8% 

Ravleigh  (T-G  Reg) 

21 

214.7% 

470.6% 

Rayleigh  MM  AC 

10 

135  8% 

282  1% 

Beta 

25 

1006.3% 

8606  8% 

OY95  Results  for  Electronic  Warfare  Data  0%  to 
25%  Complete 

Score 

MAPF|  SHAPE 

CPI  3 

8 

23  9%{  19.5% 

CP!o 

9 

23.4°  J  20.3% 

CPIcum 

0 

23  8°.'  20.0% 

S  P  j  3- 

18 

24  4% 

23  9% 

SP16 

10 

22  8% 

22.2% 

SPlcum 

2 

21  4% 

18.9% 

SC  13 

21 

28.1% 

24.0% 

SO 

18 

24  7% 

23.3% 

SO  cum 

!  12 

24  1% 

20  2% 

CPI 6  \  SPicum 

1  1 

23  7% 

20  6% 

Composite 

5 

22  8% 

19  6% 

Ravleigh  d  m  Reg  i 

25 

141  2% 

1  16  9% 

Ravleigh  >  1-G  Regi 

% 

1119% 

80.7% 

Ravleigh  MM  AC 

27 

422  8% 

564  0% 

Beta 

29 

4347  9% 

18264.4% 

CY95  Results  for  Electronic  Warfare  Data  >50% 
to  75%  Complete 

Score 

MAPI- 

SHAPE 

CPI  3 

4 

12.6°  o 

12.0% 

CP  16 

*> 

12.5% 

1  1  9% 

CPIcum 

8 

12.8% 

12  4% 

SP13 

17 

15.1% 

13  1% 

SPio 

12 

13.5% 

12.4% 

SPlcum 

10 

1 3.8% 

1  1  9% 

SO  3 

19 

15  8% 

14  8% 

SOo 

12 

13.4% 

12  9% 

SClcum 

1  1 

13.6% 

12.2% 

CP16  x  SPicum 

10 

13.1% 

12  4% 

Composite 

8 

12.9% 

12.3% 

Rayleigh  i  Lin  Reg) 

27 

233.2% 

815  8% 

Ra\  leigh  (T-G  Reg) 

25 

167.3% 

384  4% 

R  as  lei  eh  MMAL 

23 

1 06  6°  o 

1 16  7% 

Beta 

— ~1 _ 1 

57  5% 

62  8% 

Best  Performing  Yiethod(s) 

Overall  SPlcum 


0%  -  25% 
>25%  -50% 
>50%  -75% 
>75%  -  1 00% 


SPlcum 

SPlcum 

CP16 

SPi3  SPlcum 


CY 95  Results  for  Electronic  Warfare  Data  >25% 
to  50%  C  omplete 

Score 

MAPE 

SDAPE 

CP13 

8 

1 7.8% 

13.3% 

CP16 

7 

1 7  4% 

134% 

CPIcum 

8 

17.6% 

13  4% 

SP13 

13 

19  9% 

13  9% 

SPI6 

14 

19  9% 

14  1% 

SPicum 

5 

18.4% 

12  1% 

SC  13 

17 

20.4% 

1 4  3% 

SC  16 

17 

20. 1% 

1 4.4% 

SClcum 

7 

18.5% 

12  2% 

CP16  x  SPlcum 

8 

18.5% 

12.4% 

Composite 

6 

17  6% 

13.1% 

Ravleigh  (Lin  Reg) 

26 

3735  4% 

18689.8% 

Ravleigh  (T-G  Reg) 

23 

269.9% 

345.0% 

Rayleigh  MMAE 

20 

131  1% 

125.5% 

Beta 

23 

285.0% 

334.7% 

CY95  Results  for  Electronic  Warfare  Data  >75% 
to  100%  C  omplete 

Score 

MAPF 

SDAPE 

CP13 

17 

8.3% 

7.8% 

CPI6 

13 

8.1% 

7.0% 

CPIcum 

9 

7.4% 

6.0% 

SP13 

3 

6  7% 

5.8% 

SP16 

8 

7.2% 

6.7% 

SPlcum 

3 

7.1% 

5.7% 

SC  1 3 

16 

8.1% 

9.0% 

SC  16 

20 

8  5% 

9.3% 

SClcum 

10 

7.5% 

6.0% 

CP16  x  SPlcum 

15 

8  2% 

7  1% 

Composite 

7 

7.3% 

5.9% 

Rayleigh  (Ltn  Reg) 

26 

75.7% 

248.7% 

Ravleigh  (T-G  Reg) 

28 

245  8% 

602.7% 

Ravleigh  MMAE 

24 

37  9% 

53  4% 

Beta 

22 

1 1.5% 

29  6% 
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|  TV  Results  for  All  Helicopter  Data  j 

Score 

MAPE 

SDAPE 

CPI  3 

1  1 

14.1% 

14  4% 

CHIP 

4 

1  1  4% 

1 1  3% 

CPlcum 

7 

12.0% 

1 1  7% 

S  P 13 

18 

16  3% 

26  1% 

SPI6 

8 

1 1.8% 

16  0% 

SPIcum 

14 

13.8% 

26  2% 

SC  13 

22 

21  5% 

31  0% 

SC  IP 

12 

14  0% 

19  7% 

SCI  cum 

19 

1  5  0°  o 

27.8% 

CPIo  \  SPIcum 

15 

14  2% 

19  9% 

Composite 

2 

1  1 .0% 

10  7% 

Ra> ieigh  (i.m  Reg.; 

24 

1 52  9% 

405  3% 

Ravleigh  (  I  -G  Reg  ) 

26 

171  2% 

696.7% 

Ravleigh  MMAE 

N  A 

0  0% 

0  0% 

Beta 

28 

1506  2% 

8369  6% 

IA  Results  for  Helicopter  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

8 

24.0% 

1 8  9% 

CP  jo 

5 

21.5% 

1 4  2% 

CPlcum 

5 

22.9% 

13.2% 

SP13 

19 

33.1% 

42  3% 

SPI6 

10 

25.3% 

26  3% 

SPIcum 

18 

30.6% 

44.2% 

SC  13 

24 

42  9% 

46  9% 

scio 

15 

31  4% 

31  6% 

SC  I  cum 

22 

33  5% 

46  1% 

CPio  x  SPIcum 

14 

30  1% 

33  7% 

Composite 

2 

19  7% 

12  5% 

Ra\  ietgh  ( 1  in  Reg  i 

108.6% 

\2  2% 

Ra\  leigh  f'l-G  Reg  > 

18 

105.9% 

26  6% 

Ra\  Ieigh  MM  A I 

N  A 

0.0% 

0  0% 

Beta 

28 

3774.7% 

13170.8% 

TV  Results  for  Helicopter  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

rpi3 

12 

9  7% 

7.5% 

CPI6 

7 

8.0% 

6.0% 

CPlcum 

3 

7  7% 

5  3% 

SPI3 

18 

11.1% 

9.0% 

SPIo 

12 

9  5% 

7  8% 

SPIcum 

4 

7.6% 

5  9% 

SCI  3 

21 

12.8% 

10.8% 

SCio 

17 

9  8°  o 

10.0% 

SC  lain: 

9 

8.3% 

6.9% 

CPI 6  x  SPIcum 

16 

9  9% 

8  9% 

Composite 

3 

7  6% 

5.4% 

Ravleigh  ( i.m  Reg » 

26 

107.3% 

1 12  8% 

Ra\  ieigh  i  T-(i  Reg ; 

2o 

i  16  9% 

97.7% 

Ra>  Ieigh  M MAS 

N  A 

0  0% 

0.0% 

Beta 

31  4% 

32  2% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


Composite 

SPIcum 

CPlcum  Composite 
SP16  SPIcum 


TY  Results  for  Helicopter  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

16 

17.3% 

r~"  13.3% 

CPJ6 

12 

15.6% 

12.0% 

CPlcum 

9 

15.5% 

11.0% 

SP13 

13 

17  2% 

11.1% 

SP16 

7 

1 5.2% 

10  7% 

SPIcum 

3 

13  9% 

10.5% 

SCI3 

21 

22.7% 

16.9% 

SC  16 

17 

18.1% 

13  3% 

SCIcum 

9 

13  6% 

14.5% 

CPI 6  \  SPIcum 

16 

16  9% 

14.7% 

Composite 

5 

14.9% 

10  6% 

Rayleigh  (Lin  Reg) 

25 

315.5% 

842.7% 

Ravleigh  (T-G  Reg) 

23 

102.3% 

67  9% 

Ravleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

27 

2696  2% 

10980.3% 

TY  Results  for  Helicopter  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

17 

7  0% 

69% 

CP16 

11 

5.5% 

5.2% 

CPlcum 

6 

4  3% 

3  9% 

SPI3 

10 

4.3% 

5.6% 

SP16 

3 

3  6% 

3  9% 

SPIcum 

3 

3.9% 

3.8% 

SC  13 

19 

7.1% 

12  3% 

SC  1 6 

7 

4.2% 

4.6% 

SCIcum 

8 

4.9% 

4.3% 

CP16  x  SPIcum 

14 

6  2% 

5  8% 

Composite 

4 

4  2% 

3.8% 

Ravleigh  ( Lin  Reg) 

21 

124. 1% 

260.6% 

Ravleigh  (T-G  Reg) 

23 

265.5% 

1101.5% 

Rayleigh  MMAE 

N  A 

0  0°  o 

0.0% 

Beta 

13 

4  9% 

6.8% 
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CY95  Results  for  All  Helicopter  Data  | 

Score 

MAPE 

SDAPE 

CP!  3 

8 

13  9% 

15.0% 

CPA 

5 

11.9% 

10  0% 

CPIcum 

5 

12.1% 

9  9% 

SP13 

17 

18  2% 

30.9% 

SPA 

9 

1 3  9% 

1 8.3% 

SPlcum 

14 

15  9% 

31.0% 

SC  i? 

21 

23.6% 

36  7% 

SCIC* 

13 

16  7% 

23.1% 

SC  k  urn 

18 

17  4% 

32  9% 

CP16  \  SPlcum 

13 

16.6% 

23  2% 

Composite 

o 

11.2% 

8  9% 

Raxieigh  (Lin  Reg) 

23 

138.0% 

402.1% 

Raxieigh  (T-G  Reg! 

27 

409  7% 

3594  8% 

Raxieigh  MMAL 

25 

357.2% 

588  4%, 

Beta 

29 

2043.6% 

10289  6% 

CY95  Results  for  Helicopter  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI.) 

8 

19.8% 

21  6% 

CPA 

4 

1 6  7 % 

12.4% 

C’Pkum 

6 

18.0% 

12  6% 

SPIA 

19 

35.1% 

52  8% 

SPA 

1  i 

27.1% 

33  0% 

Si'icum 

17 

32.5% 

54  5% 

SC  13 

24 

47  0% 

58  8% 

SO  6 

15 

34  3% 

40.2% 

SCicum 

22 

35  8% 

56  9% 

CP16  \  SPlcum 

15 

32.9% 

41  6% 

Composite 

"i 

15.1% 

1 1  2% 

Raxieigh  ( Lin  Reg  i 

22 

1  10.5% 

45  4% 

Rax  lei  gb  .T-G  Reg  ) 

17 

107.0% 

28  8% 

Raxieigh  MMAE 

28 

724  9% 

937  9% 

Beta 

30 

5433  7% 

16285  8% 

CY95  Results  for  Helicopter  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

8 

9.5% 

6.2% 

CPA 

4 

8.5% 

5.8% 

CPic  urn 

n 

8.4% 

5  3% 

SPA 

15 

1  1.5% 

7  9% 

SPin 

1  1 

9.8% 

7.2% 

SPL  urn 

4 

8.4% 

6  1% 

SC  13 

19 

12.0% 

10  5% 

SC  16 

15 

10.0% 

9  8% 

SCicum 

8 

9  2°o 

6  7% 

CPA  \  SPlcum 

15 

1 0  6% 

8  5% 

Composite 

"x 

8  4% 

5.3% 

Raxiemh  ii.m  Reg) 

23  : 

124.2% 

147.3% 

Rax  lei gh  <  T-G  Reg) 

27 

1248.9% 

7162  7% 

Raxieig’n  MMAE 

25 

362  4% 

569.9% 

Beta 

3  1  5% 

32  0% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 


Composite 
Composite 
CPIcum  Composite 


>75%  -  100%  Beta 


CY95  Results  for  Helicopter  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

14 

15  8% 

12  2% 

CP16 

10 

14.0% 

10  9% 

CPIcum 

5 

13  4% 

10  0% 

SP13 

12 

15  2% 

12.0% 

SPI6 

7 

1 3  8% 

10  6% 

SPlcum 

5 

12  2% 

10.8% 

SC  1 3 

21 

22  0% 

18.6% 

SC  16 

17 

17  9% 

14  5% 

SCicum 

13 

13  8% 

14.6% 

CP16  \  SPlcum 

18 

17.3% 

o' 

Composite 

3 

13  0% 

9.5% 

Ravleigh  (Lin  Reg) 

26 

265  7%. 

820  4% 

Rayleigh  (T-G  Reg) 

23 

141  2% 

267.3% 

Rayleigh  MMAE 

26 

429  8% 

404.1% 

Beta 

29 

3156  1% 

13002  1% 

CY95  Results  for  Helicopter  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

18 

10  7% 

11.9% 

CPI6 

14 

10  1% 

8  8% 

CPIcum 

8 

9  1% 

7  0% 

SP13 

8 

9.2% 

6  9% 

SPA 

4 

8  7% 

6.7% 

SPlcum 

4 

8.9% 

6.6% 

SC  13 

20 

11.0% 

12.2% 

SC16 

14 

9  7% 

8  9% 

SCicum 

1  1 

9.4% 

7  1% 

CP16  x  SPlcum 

15 

10.5% 

8  8% 

Composite 

7 

9  1% 

6.9% 

Rayleigh  (Lin  Reg) 

24 

96.9% 

283.7% 

Rayleigh  (T-G  Reg) 

25 

128.6% 

130.4% 

Ravleigh  MMAE 

23 

107.9% 

123  4% 

Beta 

4  4% 

6.6% 
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Results  for  Other  Missile  Contracts  Usintz  Non-Inflation  Adjusted  Data 


|  TV  Results  for  All  Other  Missile  Data  | 

Score 

MAPE 

SDAPE 

CPI  3 

14 

11.4% 

12  6% 

CPio 

7 

10.6% 

1 1 .5% 

CPIcum 

7 

10  5% 

116% 

SP13 

15 

1 2  6% 

119% 

SPIo 

10 

1 1  5% 

1 1  2% 

SPlcum 

5 

10  7% 

9  4% 

SC  13 

19 

14  4% 

14  6% 

SC  IP 

15 

12  !% 

12.4% 

SC]  cum 

8 

1  1  1% 

11.2% 

CPIp  \  SPlcum 

8 

11  1% 

1  1.2% 

Compos  lie 

3 

10.3% 

10  8% 

Rayleigh  (Lin  Real 

2 ! 

134  0% 

287.5% 

RaUeigh  d-G  Reg: 

-J  > 

235.6% 

681  7% 

Ravleigh  MMAE 

N  A 

0.0% 

0.0% 

He  la 

25 

1070  6% 

6820  5% 

TV  Results  for  Other  Missile  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPP 

17 

22  6% 

17  6% 

CP  16 

14 

21  7% 

1 6  0% 

CPIcum 

1 0 

20.9% 

15  8% 

SPI3 

8 

n  j  o 

13  6% 

SP16 

6 

21  0% 

13  3% 

SPlcum 

1 7  6% 

1  1  3% 

SC  13 

18 

24.9% 

17.5% 

SCIr 

14 

23.3% 

1  5  7% 

SCI  cum 

8 

21  0°  o 

14  2% 

CPlo  S  Pi  cum 

10 

n  j  -to 

14.5% 

Composite 

6 

19  9%. 

14.2% 

Ravleigh  Ci  in  Reg! 

21 

108.2% 

46.5% 

Rav icigh  iT-G  Reg) 

25 

154.1% 

228  8% 

Rav leign  MMAE 

N  A 

0  0% 

0  0% 

Beta 

25 

3303  7% 

12510  4% 

TV  Results  for  Other  Missile  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPC 

8 

8  3% 

5  8% 

CP  16 

2 

7.2% 

4  7% 

CPIcum 

7 

7.6% 

5  9% 

SPI3 

17 

11.1% 

9  1% 

SPIo 

15 

9.7% 

8.3% 

SPlcum 

1  1 

8.9% 

6.3% 

SC  13 

19 

1  1 .5% 

12.2% 

SOI 9 

13 

9  0% 

7.7% 

SCI  cunt 

6 

7  8% 

5.5% 

CPI 6  x  SPlcum 

7.2% 

4  9% 

Composite 

7 

7.6°  o 

5  9% 

Ravleigh  (Lm  Reg) 

23 

266.9% 

$43.2% 

Rav  leigh  ( T-G  Reg) 

2  ^ 

624.9% 

1444.3% 

Ra\  leigh  MMAE 

N  A 

0  0% 

0  0% 

BcU 

2 ! 

65.0°  o 

69  9®-o 

Best  Performing  Method(s) 

Overall  Composite 

0°  o  -  25%  SPlcum 

>25%  -  50%  CPI3 

>50%  -  75%  CP16 

>75%  -100%  CPIcum 


TV  Results  for  Other  Missile  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

2 

9  8% 

7  4% 

CPI6 

4 

10.3% 

7  7% 

CPIcum 

7 

10.5% 

9.0% 

SP13 

19 

1 5  4% 

113% 

SP16 

17 

15.0% 

10.6% 

SPlcum 

11 

13  8% 

9  3% 

SCI3 

21 

15.7% 

1  1.7% 

SC16 

15 

14  7% 

10.3% 

SCI  cum 

12 

12  9% 

10.0% 

CP16  x  SPlcum 

10 

12  8% 

9.7% 

Composite 

6 

10  5% 

8.9% 

Rayleigh  (Lin  Reg) 

23 

123  0% 

106.7% 

Ravleigh  (T-G  Reg) 

25 

164.3% 

333  6% 

Ravleigh  MMAE 

N.  A 

0  0% 

0.0% 

Beta 

27 

961  8% 

4411.2% 

TV  Results  for  Other  Missile  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

14 

5  3% 

5.3% 

CPI6 

17 

5  4% 

5.7% 

CPIcum 

3 

4  7% 

3  6% 

SP13 

1  1 

5.1% 

5.2% 

SPI6 

5 

4  6% 

4.0% 

SPlcum 

1  1 

5  ~>% 

4.2% 

SC13 

21 

6.5% 

8  1% 

SC16 

13 

5  1% 

5.6% 

SClcum 

7 

4  9% 

3  8% 

CP16  x  SPlcum 

19 

5  8% 

6.0% 

Composite 

5 

4  8% 

3  7% 

Ravleigh  (Lin  Reg) 

26 

95  4% 

254.7% 

Ravleigh  (T-G  Reg) 

26 

130.6% 

140  2% 

Ravleigh  MMAE 

N  A 

0  0% 

00% 

Beta 

23 

14.5% 

28.5% 
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Results  for  Other  Missile  Contracts  Using  Inflation  Ad)usted  Dam 


|  CY95  Results  for  All  Other  Missile  Data  ] 

Score 

MAPE 

SDAPE 

CPI  3 

12 

11.3% 

12  1% 

CP16 

7 

10  6% 

10.8% 

CPicum 

7 

10.3% 

111% 

SPI3 

14 

12.6% 

11.5% 

SPIP 

9 

1  1.3% 

10  9% 

SPIcum 

4 

10.5% 

9.0% 

SC  1 5 

17 

14.5% 

14.2% 

SC  It* 

14 

12  2% 

12  1% 

SCI  cum 

10 

1 1  3% 

11  1% 

CP  It'  \  SPIcum 

9 

114% 

10  8% 

Composite 

3 

10  0% 

10.2% 

Ra\  leigh  ( i.in  Reg} 

19 

127  1% 

4 1 2  2% 

Ra\ ieigh  (T-G  Reg) 

23 

438.2% 

4139  1% 

Ra\  leigh  MMAE 

2) 

239  0% 

557.0% 

Bela 

25 

2230  1% 

17675  3% 

CY95  Results  for  Other  Missile  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPE3 

17 

19.4% 

18  4% 

CPU* 

i 

1 8.5% 

16.7% 

CPicum 

9 

17  8% 

16.3% 

SP13 

9 

1  8.9% 

14.3% 

SP16 

6 

1  8  4% 

13  8% 

SPIcum 

2 

1 4.8% 

1 1  6% 

SC' 13 

18 

23  9% 

1 8  0% 

SC  16 

15 

21.5% 

16.3% 

SCI  cum 

I  ! 

1 9  2% 

15  0% 

CPI 6  x  SPIcum 

12 

19  2% 

15.1% 

Compos  ue 

5 

16  8% 

14  3% 

Ra\ leigh  i  Lip  Reg  i 

22 

107  4% 

60.1% 

Ra\ leigh  i  T-G  Reg  > 

p-> 

1  1  1.2% 

52.0% 

Rayleigh  M M  A E 

^  s 

358  0% 

626  5% 

Beta 

"7 

6836  4% 

33071.5% 

CY95  Results  for  Other  Missile  Data  >50%  to 
75%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

6 

8.2% 

5.6% 

CP16 

3 

7.3% 

5.0% 

CPicum 

8 

7.6% 

6.2% 

SPI3 

17 

1 1.0% 

8.2% 

SP16 

15 

9.5% 

OO 

o 

SPIcum 

10 

9.1% 

6  0% 

SC  1 3 

19 

11.1% 

10.7% 

SC  1 6 

12 

8  8% 

7.2% 

SCI  cum 

6  ! 

7.6% 

5.9% 

CPI6  \  SPIcum 

n 

7  1% 

5  0% 

Composite 

9 

7.6% 

6.3% 

Rayleigh  (Em  Reg  i 

24 

126  8% 

148.1% 

Ra\  leigh  1 7 -G  Reg ) 

27 

215  7% 

249  4% 

Rayleigh  M MAE 

24 

187  7% 

118.0% 

Beta 

21 

61.4% 

66  5% 

Best  Performing  Method(s) 


Composite 

SPIcum 


Overall 
0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


CPI3 

CPI6  \  SPIcum 
SP16 


CY95  Results  for  Other  Missile  Data  >25%  to 
50%  Complete 

Score 

MAPE 

SDAPE 

CP  13 

2 

9.4% 

7.0% 

CPI6 

5 

9.9% 

7.4% 

CPicum 

5 

9.8% 

8.9% 

S  FI  3 

18 

15  5% 

117% 

SP16 

15 

146% 

11.5% 

SPIcum 

10 

13  3% 

9.9% 

SCI3 

20 

16  0% 

12.2% 

SC  16 

15 

14  8% 

11.1% 

SCI  cum 

10 

12  7% 

10  6% 

CP16  x  SPIcum 

10 

12  9% 

10.2% 

Composite 

5 

9  8% 

8.9% 

Rayleigh  (Lin  Reg) 

22 

138.3% 

148  7% 

Rayleigh  (T-G  Reg) 

24 

231  8% 

827  4% 

Rayleigh  MMAE 

26 

474.8% 

1001  6% 

Beta 

28 

2162.9% 

9756  8% 

CY95  Results  for  Other  Missile  Data  >75%  to 
100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

15 

7.7% 

6.6% 

CPI6 

12 

7  6% 

6  3% 

CPicum 

6 

6  6% 

5.5% 

SPI3 

9 

6  7% 

6.1% 

SPI6 

3 

6.0% 

5.3% 

SPIcum 

4 

6.7% 

5.1% 

SC13 

15 

7.6% 

7.5% 

SCI6 

7 

6.6% 

5  9% 

SC  I  cum 

8 

6.8% 

5.5% 

CP16  x  SPIcum 

15 

7  9% 

6  4% 

Composite 

5 

6.6% 

5.4% 

Rayleigh  (Lin  Reg) 

23 

132.0% 

639  2% 

Rayleigh  (T-G  Reg) 

25 

762.3% 

6282.5% 

Rayleigh  MMAE 

21 

87  9% 

92  4% 

Beta 

19 

14.0% 

28.0% 
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Results  for  Satellite  Contracts  Using  Non-Inflation  Adjusted  Data 


j  TV  Results  for  AH  Satellite  Data 

“ 

Best  Performing  Method(s) 

Overall  SPI6  SPicum 

03b  -  253o  sc  16 

>253o  -  50%  Composite 

>503o  -  75%  SPicum 

>75°/o  -  1003o  SPI6 

Score 

MAPE 

SDAP1 

" 

CP  [3 

18 

10  4° 

12  6° 

CPI6 

9 

7.2° 

1 1  59 

CPfcum 

!  1 

7.4° 

116°' 

SPI3 

15 

8  4° 

1 19°/ 

SP16 

4 

6.2°( 

1 1 .23 

SPicum 

4 

6  7% 

9  4°/ 

SC  13 

20 

13  7°c 

14  63 

SCiO 

14 

7.79  c 

12  4° 

SClcum 

5 

6.5°c 

1 1 .29 

CPto  \  SPicum 

1  1 

7.0% 

1  1.2% 

Composite 

6 

7  0% 

10.8% 

Rayleigh  ( I. in  Reg ) 

22 

82  9% 

287.5% 

Ra\  lei  eh  (T-G  Reg) 

24 

464  2% 

681  7° o 

Ra\  leigh  MMAF 

N  A 

0.0% 

0  0% 

Beta 

26 

617.3% 

6820.5% 

1  TV  Results  for  Satellite  Data  0%  to  25% 

1  TV  Results  for  Satellite  Data  >25%  to  50% 

Com 

plete 

Complete 

Score 

MAPE|  SDAPE 

|  Score 

!  MAPF 

SDAPF 

CP  13 

12 

17.5% 

7  9% 

CP13 

16 

T  10  6% 

1 1.5% 

CP!  6 

12 

1 4.4° o 

9  73b 

CPto 

7 

6  4% 

4  9% 

C  PI  cum 

10 

14.8% 

7  8% 

CPIcum 

5 

6.530 

2.8% 

SPi3 

12 

14.0% 

9  9% 

SP13 

!  14 

1  1.0% 

7.6% 

SP1G 

0 

11.0% 

9  1  °o 

SP16 

8 

9  1% 

4.5% 

SPicum  ~] 

7 

13  0% 

7  9% 

SPicum 

8 

7  8% 

4  8% 

SC  13 

17 

20.9% 

15.1% 

SC  1 3 

19 

17  6% 

1  7.03o 

SClf' 

4 

12.5% 

7  73o 

SCI6 

14 

9  7% 

9  3% 

SCI  cum 

7 

13  0% 

7  93o 

SClcum 

5 

3.530 

4.9% 

CP  to  \  SPicum 

5 

13  6% 

7  6% 

CPto  x  SPicum 

I  1 

7.8% 

7.8% 

C  ompositc 

7 

14  U% 

7  7% 

Composite 

3 

6.3% 

2  8% 

Rav  leigh  i  Lm  Reg ) 

2(1 

91.0% 

27  6% 

Ravleigh  (Lin  Reg) 

21 

222. 9% 

326  2% 

Rayleigh  (T-G  Reel 

2c 

79.3% 

52  6% 

Ravleigh  (T-G  Reg) 

25 

1518  9% 

3888.8% 

Rav  leigh  MM  At 

N  A 

0.0% 

0.03  b 

Ravleieh  MMAF 

N  A 

0.0% 

0.03o 

Beta 

23 

2768  8% 

2353  4% 

Beta 

21 

86  1% 

61  1% 

1  TV  Results  for  Satellite  Data  >50%  to  75%  j 

TV  Results  for  Satellite  Data  >75%  to  100%  [ 

Com 

□  lete 

Complete 

i 

Score 

MAPI: 

SDAPE 

i 

Score 

MAPE 

SDAPE 

CPI  3 

17 

7.2% 

4.2% 

CPI3 

23 

7  7% 

7  03b 

CPto 

1  1 

4.5% 

3  930 

CPto 

18 

5  930 

4  13o 

CPI  cum 

7 

4  0% 

2  43b 

CPIcum 

12 

5.3% 

2.4% 

SP13 

17 

6  7% 

5  1% 

SPI3 

5 

4,1% 

2.03o 

SP16 

“7 

3  8% 

2.530 

SPJ6 

3 

3  4% 

1.9% 

SPicum 

-> 

2.6% 

1.93  b 

SPicum 

7 

4  4% 

2.23o 

SC13 

23 

12.5% 

6.83o 

SCI3 

23 

7  63b 

7. 13o 

sc  to 

16 

6.2% 

5.33b 

SCI6 

13 

5  03o 

3  4°  o 

SClcum 

1  1 

4  7% 

2  8% 

SClcum 

14 

5  4% 

2  9°b 

CPto  \  SPicum 

18 

6  3% 

5  630 

CPto  x  SPicum 

20 

6.330 

4.330 

Composite 

4 

3.7% 

2.2% 

Composite 

10 

5.1%. 

2.3% 

Rayleigh  1 1 .in  Rem 

26 

31  7% 

27.6% 

Rayleigh  (Lin  Reg) 

26 

1 3  63b 

14  4% 

Rayleigh  (T-G  Reg) 

28 

6 1 .4° o 

47.6% 

Rayleigh  (T-G  Reg) 

28 

52  8% 

46  7% 

Ra\ leigh  MMAF- 

N  A 

0  0% 

0  0% 

Ravleigh  MMAF 

N.  A 

0.0% 

0.0% 

Beta  i 

23 

12  6% 

6.23b 

Beta 

8 

3.0% 

3  1 0  o 
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Results  for  Ship  Contracts  Using  Non-Inflation  Adjusted  Data 


|  TV  Results  for  All  Ship  Data  f 

Score 

MAPE 

SDAPE 

CPC 

8 

8.1% 

8  6% 

CPio 

5 

7.3% 

7  3% 

CPlcum 

5 

6  6% 

7  6% 

SPI3 

20 

14  9% 

18  0% 

SPlft 

15 

12.6% 

14  4% 

SPlcum 

1  I 

11  1% 

14.0% 

SC '  j  3 

-> 

16  1% 

20.0% 

5C’j6 

17 

13  5% 

15.1% 

SClc  urn 

15 

1  1  4% 

15.2% 

C  P  16  v  SPlcum 

12 

1 1  7% 

13.7% 

Composite 

o 

6.5% 

6  7% 

Ravleigh  d  m  Reg) 

28 

517  5°  o 

3137  5% 

Ravleigh  i  T-G  Reg  i 

26 

161  0% 

1  79  6% 

Ra\  leigh  MMAE 

N  A 

0.0% 

0  0% 

Beta 

24 

78  0% 

135  3% 

TV  Results  for  Ship  Data  0%  to  25%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

10 

13  4% 

1 2  4% 

CPP 

7 

12.4% 

10.5% 

CPlcum 

7 

1 1  .3% 

12  0% 

9  Pi ; 

21 

26  4% 

23  0% 

SPj6 

14 

'»')  )  0  0 

1  8  3% 

SPlcum 

16 

23  2% 

19  5% 

St.'!  3 

23 

28  6% 

23  6% 

SC  in 

15 

24  4% 

18  1% 

Si  Rum 

io 

25  4% 

21  8% 

CP16  x  SPlcum 

15 

25  4°-o 

17.6% 

Composite 

3 

11  1% 

9  7% 

Ravleigh  f  1  .in  Reg) 

14 

103  3% 

9.9% 

Ravleigh  El  -G  Reg  i 

j 

104  5% 

5  9% 

Rav  leigh  V1MAL 

N  A 

0.0%. 

0  0% 

Beta 

27 

203  1% 

193  6% 

TV  Results  for  Ship  Data  >50%  to  75%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

1  1 

5.7% 

4  6% 

CPio 

2 

3.8% 

2  5% 

CPlcum 

4 

4  1% 

2  6% 

SPI3 

19 

1 0  8°  o 

10  4% 

S  P 1 6 

14 

8  6% 

5  7% 

SPlcum 

0 

4.9% 

4.4% 

SC' 13 

17 

10  7% 

9.3% 

SCIo 

14 

8  4% 

6.0% 

SCI  cum 

7 

4  8% 

3  4% 

CPio  \  SPlcum 

s 

4  1% 

3.2% 

(  ompusite 

4 

4  1% 

2.6% 

Rav  leigh  (  Lin  Reg) 

25 

223  4% 

265  7% 

Rev  leigh  i  i-G  Reg) 

23 

1 79  3% 

137.0% 

Rav leigli  MMAI: 

N  A 

0  0% 

0  0°  o 

Beta 

21 

17.0% 

215% 

Best  Performing  Method(s) 

Overall  Composite 


0%  *  25% 
>25%  -  50°.  o 
>50%  -  75% 
>75%  -  100% 


Composite 
CPlcum  Composite 
CP  16 

CPlcum  Composite 


TV  Results  for  Ship  Data  >25%  to  50%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

8 

8.5% 

6.7% 

CPI6 

6 

8.2% 

5.3% 

CPlcum 

3 

6  7% 

4.5% 

SP13 

19 

17  2% 

20  7% 

SP16 

15 

15  5% 

16.8% 

SPlcum 

12 

IE  6% 

8.5% 

SC  13 

21 

19  2% 

27  0% 

SC  16 

17 

1 6  4°  o 

18  2% 

SCI  cum 

10 

9  9% 

7.3% 

CP16  x  SPlcum 

13 

116% 

9.1% 

Composite 

3 

6.5% 

4.7% 

Ravleigh  { Lin  Reg) 

24 

122.7% 

33  9% 

Rav leigh  (T-G  Reg) 

->? 

127  2% 

59.9% 

Rayleigh  MMAE 

N  A 

0  0% 

0  0% 

Beta 

24 

81.0% 

58  6% 

TV  Results  for  Ship  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

10 

4  8% 

3.9% 

CP16 

9 

4  6% 

3  9% 

CPlcum 

3 

4.0% 

3  !% 

SPI3 

15 

5.7% 

5.2% 

SPI6 

13 

5  1% 

5.0% 

SPlcum 

5 

3  9% 

3.4% 

SCI3 

20 

6  7% 

7.4% 

SC  16 

18 

5  8% 

6.4% 

SCicum 

6 

4  !% 

3.3% 

CP16  x  SPlcum 

1 1 

4.8% 

4.1% 

Composite 

3 

3  9% 

3  2% 

Ravleigh  {Lin  Reg) 

24 

1233.9% 

5355  5% 

Ravleigh  (T-G  Reg) 

196  7% 

267  8% 

Ravleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

13 

4  6% 

5  8% 
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CY95  Results  for  All  Ship  Data  j 

Score 

MAPE 

SDAPE 

CP!  3 

8 

8.5% 

10.0% 

CPI6 

6 

7.3% 

8  1% 

CPIcum 

4 

6.9% 

7.6% 

SPI3 

IP 

14  6% 

18  8% 

SPi6 

14 

12  2% 

15.3% 

SPlcum 

10 

11  4% 

14.9% 

SC!.' 

21 

15.9% 

21  7% 

SC  !o 

17 

13.3% 

16  4% 

SCI  cum 

14 

1 1  9% 

16  4% 

CPir  \  SPieum 

14 

12  0% 

1  5  5% 

Composite 

Y 

6.4% 

5  8% 

Ra\ieigh  ( Lin  Reg) 

27 

2 19.2% 

740. 1  % 

Ra\ ieigh  (T-G  Reg) 

20 

397.1% 

1244  2% 

Ra\  leigh  MV1AL 

n 

97  2% 

1  12.8% 

Beta 

25 

99.8% 

239  4% 

Best  Performing  Method(s) 

Overall  Composite 


0%  *  25°  o 
>25%  -  50% 
>50%  -  75% 
>75%  -  100% 


Composite 

Composite 

CPI6 

SP16  SPlcum  Composite 


C V 95  Results  for  Ship  Data  0%  to  25%  Complete) 


Score 

MAPH 

SDAPE 

CPI  ? 

9 

1 4  4% 

15  0% 

CPI6 

7 

12  4% 

12  4% 

CPlcirm 

s 

12  1% 

116% 

SP13 

21 

27.0% 

24.4% 

SPio 

12 

22  4% 

20.2% 

SPlcum 

15 

23  3% 

21.7% 

SCI  3 

23 

30.0% 

26  5% 

sc  it* 

15 

25.2% 

21  4% 

SC  Icum 

18 

26  4% 

24.3% 

CP16  \  SPlcum 

1  7 

26.5% 

21  4% 

Compos  He 

10.1‘V 

8  1% 

Run Ieigh  (Lin  Reg i 

is  ! 

!  06  0% 

19  4% 

Ru\ Ieigh  < 1  -G  Reg) 

15 

106  1% 

9  4% 

Rav Ieigh  MMAL 

27 

128  3% 

193  5% 

Beta 

29 

275  8% 

392.4% 

CY95  Results  for  Ship  Data  >50%  to  75% 
Complete 

Score 

mapl 

SDAPE 

CP13 

12 

4.9% 

4  4% 

CP  16 

2 

3.4% 

. 

CPIcum 

6 

3  7% 

9% 

SPI3 

18 

9.3% 

9  1% 

SP16 

15 

6.9% 

6  4% 

SPlcum 

1  1 

4.9% 

4.0% 

SC  13 

18 

8  4% 

9.2% 

SCit' 

14 

6  9% 

6  3% 

SC  Icum 

9 

4  3% 

3.2% 

CPI6  \  SPlcum 

4 

3  5% 

2  9% 

Composite 

6 

3  6% 

3.0% 

Ra\  leigh  ( Lin  Reg) 

25 

171  2% 

90.4% 

Rav  leigh  (T-Ci  Reg  > 

27 

336  9% 

639  9% 

Ra\ !e: gh  MMAL. 

23 

83.8% 

58.5% 

Beta 

21 

17.6% 

21.8% 

CY95  Results  for  Ship  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

7 

8  6% 

7.1% 

CP16 

5 

7.8% 

5.6% 

CPIcum 

3 

6.1% 

4.6% 

SPI3 

19 

16.9% 

21.9% 

SP16 

15 

14  6% 

1 7.6% 

SPlcum 

11 

10.6% 

9.6% 

SCI3 

21 

18  9% 

28.7% 

SC!  6 

17 

16  1% 

18  8% 

SClcum 

9 

9.8% 

7  4% 

CP16  \  SPlcum 

13 

112% 

9.7% 

Composite 

2 

6  1% 

4.4% 

Rayleigh  (Lin  Reg) 

24 

132.0% 

52  5% 

Rayleigh  (T-G  Reg) 

29 

151  3% 

148  9% 

Rayleigh  MMAF 

27 

145.7% 

72.5% 

Beta 

24 

86.0% 

57  4% 

CY95  Results  for  Ship  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

14 

5  6% 

4.6% 

CP16 

9 

5.4% 

4.1% 

CPIcum 

5 

5.0% 

3.6% 

SPI3 

13 

5  6% 

4.4% 

SPI6 

4 

5  1% 

3.4% 

SPlcum 

4 

5.1% 

3.4% 

SCI3 

18 

6  7% 

6.5% 

SC16 

13 

5.8% 

4  2% 

SClcum 

7 

5  1% 

3.7% 

CP16  x  SPlcum 

11 

5.5% 

4.2% 

Composite 

4 

5.0% 

3.5% 

Rayleigh  (Lin  Reg) 

22 

381  6°  J 

1262.3% 

Rayleigh  (T-G  Reg) 

24 

724  5% 

1959  3% 

Ravleigh  MMAE 

20 

56.5% 

49.9% 

Beta 

10 

4  4% 

5.3% 
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CV95  Results  for  AH  Torpedo  Data 


Score 

MAPE 

SDAPE 

CP  13 

10 

14  7% 

14.2% 

CPlo 

6 

13  4% 

13.4% 

CPlcum 

6 

13  1% 

14  0% 

St*  1 3  1 

17 

18.2% 

16  0% 

SPiP 

13 

16.1% 

14  2% 

SPIcurn 

6 

14  8% 

12  7% 

SC  13 

20 

20.8% 

22  5% 

SOp 

17 

17  8% 

16  2% 

SCIcum 

8 

1 5.5% 

13  0% 

CPIp  x  SPicum 

10 

16.3% 

1 3  0% 

Composite 

6 

13.2% 

13  7% 

Rayleigh  { I.in  Reg  l 

26 

192.2% 

689  5% 

Ravleigh  (T-G  Reg) 

24 

138  4% 

262.0% 

Ravleigh  MMAI. 

134  1% 

198  3% 

Best  Performing  Method(s) 

Overall  CPI6  CPlcum  SPIcurn  Composite 

0%  -  25%  SPIcurn 

>25%)  -  50%  CPI3  SPIcurn  SCIcum 

>50% -75%  CPI3  CPlcum  SCI3 

>75%-  100%  SP16 


CY95  Results  for  Torpedo  Data  0%  to  25% 
Complete 


Score 

;  MAPE 

SDAPE 

CP  13 

16 

24  0% 

18.1% 

CPIp 

9 

19  6% 

17  7% 

CPlcum 

10 

21.1% 

17  5% 

SP!3 

1  1 

27.7% 

14  4% 

SPIp 

10 

24  1% 

14  8% 

SPlcum 

5 

218% 

14  4% 

SC13 

2! 

32  4% 

24  0% 

CY95  Results  for  Torpedo  Data  >50%  to  75% 
Complete 


_  fl  Score  |  MAPE  SDAP1 


SP16 

10 

1 1  1  % 

8.0% 

SPlcum 

8 

9  8% 

HHKEjfl 

SC  13 

7 

10  7% 

7  3% 

SC  16 

17 

12  8% 

9  1% 

SCIcum 

8 

10.0% 

84% 

CPlP  x  SPlcum 

15 

12.0% 

8.9% 

Composite 

8 

9.0% 

8.8% 

2p 

709  3% 

1636.1% 

Ravleigh  (T-G  Reg) 

23 

1  1 7  5% 

1 1 6  6% 

| Ravleigh  MMAE 

23 

1 14.5% 

138  7% 

Beta  ^ 

20 

13  0% 

9.9% 

CY95  Results  for  Torpedo  Data  >25%  to  50% 
Complete 


APE 


SCI6  1 

28.9% 

19.7% 

CY95  Results  for  Torpedo  Data  >75%  to  100% 
_ _  Complete 


||  Score  MAPE  SDAPE 


SPI3 


SPI6 


SPIcurn 


SCI3 _ 

SCI6 _ 

SCIcum 
CPI6  x  SPlcum 


Ravleigh  (T-G  Reg) 


Rayleigh  MMAI 


g ] 
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P  TY  Results  for  All  Development  Data  | 

|[  Score  |  MAPE 

SDAPH 

CPI  3 

16 

15  3% 

19  6% 

CPI6 

9 

13.5% 

18  9% 

CPIcum 

15 

14  7% 

19  6% 

SPI3 

9 

14.5% 

18.7% 

SP16 

3 

13.0% 

18.7% 

SPIcum 

7 

13  4% 

18  8% 

SC  1 3 

18 

15  7% 

20  3% 

SC  in 

7 

13  1% 

19  0% 

SCI  cum 

Q 

1 3.3% 

19.4% 

CP 1 6  x  SPIcum 

2 

12  6% 

1 8  7% 

Composite 

12 

14  4% 

19.4% 

RaGeigh  (Lin  RetCi 

20 

100  8% 

389. 8% 

Ravi ei. ah  tl-G  Reg  i 

178  9% 

507.4% 

Ravleigh  MMAE 

N  A 

0.0% 

0  0% 

Beta 

24 

185964. 5°  o 

3664211  4% 

TY  Results  for  Development  Data  0%  to  25% 
Complete 

|  Score 

1  mapei  SDAPFl 

CPI3 

14 

31.1 0  o 

24.0% 

CPIo 

12 

30.690 

23.5% 

CPIcum 

1  ! 

3  3.4% 

218% 

SP13 

7 

269% 

22.6% 

spin 

9 

26.8% 

24.3% 

SPIcum 

3 

25.9% 

22.3% 

SCI  3 

17 

30.6°  o 

26.0% 

SC  In 

14 

28  1% 

25  4% 

SCI  cum 

9 

27.0% 

23.5% 

CPI6  x  SPIcum 

9 

26  6% 

25.3% 

Composite 

8 

30.3% 

21.8% 

Ra\  leigh  ( Lin  Reg) 

21 

115  0% 

92.9% 

Ravleigh  1 1  -C'i  Reg) 

23 

1 36  8°  o 

198  4% 

Ravleigh  MMAF 

N  A 

■0.0% 

0  0% 

Beta 

2  s 

825464.4% 

7724578.7% 

TY  Results  for  Development  Data  >50%  to  75% 
Complete 

|  Score 

MAPE 

SDAPE 

CPI  3 

8 

1 1  3% 

10  8% 

CPI6 

5 

10.1% 

1 1.2% 

CPIcum 

1  1 

10.4% 

11.7% 

SP13 

15 

12.0% 

11.9% 

SPln 

14 

11  1% 

12  0% 

SPIcum 

8 

10.4% 

1 1.3% 

SC  S3 

19 

12  7% 

13  1% 

SC  in 

13 

10  2% 

12.1% 

SC  I  cum 

6 

9.9% 

11.5% 

CPIn  \  SPIcum 

4 

9.7% 

11.3% 

Composite 

10 

10.4% 

1 1  6% 

Rayleigh  ( Lin  Reg ) 

23 

111.8% 

322.3% 

Ravleiph  <T-G  Reg) 

i  25 

239.8% 

62 1  1  % 

Ra\  leigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

21 

60.6% 

116  7% 

Best  Performing  Method(s) 

Overall  CPI6  x  SPIcum 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPJcum 
SC  1 3 

CP16  x  SPIcum 
CPIcum  Composite 


TV  Results  for  Development  Data  >25%  to  50% 
Complete 

1  ||  Score 

MAPE 

SDAPE 

CPI3 

12 

23.4% 

24  9% 

CP16 

12 

23.1% 

25  7% 

CPIcum 

15 

23.1% 

26.5% 

SPI3 

8 

22  8% 

24.6% 

SPI6 

10 

22  79o 

25.8% 

SPIcum 

12 

22.6% 

26.6% 

SC  13 

5 

22.6% 

24.1% 

SC  16 

7 

22  3% 

25.7% 

SCIcum 

10 

20  9% 

27.4% 

CPI6  x  SPIcum 

8 

21.8% 

26  4% 

Composite 

15 

22  9% 

266% 

Rayleigh  (Lin  Rea) 

20 

250.3% 

793  7% 

Ravleiph  (T-G  Reg) 

r  22 

329.1% 

870  0% 

Ravleiph  MMAE 

NA 

0.0% 

0.0% 

Beta 

24 

1783.5% 

778 1 .9% 

TV  Results  for  Development  Data  >75%  to  100% 
Complete 

l  II  Score  |  MAPE 

SDAPE 

CPI3 

15 

4  8% 

5.0% 

CPI6 

9 

4  3% 

3.9% 

CPIcum 

2 

3  9% 

3.4% 

SPI3 

12 

4.4% 

4  3% 

SPI6 

3 

3.9% 

3.5% 

SPIcum 

6 

4.0% 

3.7% 

SCI3 

17 

5  3% 

6.4% 

SC  16 

6 

4  2% 

3  6% 

SCIcum 

5 

4.0% 

36% 

CP16  x  SPIcum 

12 

4  5% 

4.2% 

Composite 

2 

3.9% 

3  4% 

Rayleigh  (Lin  Reg) 

20 

17  8% 

23.2% 

Rayleigh  (T-G  Reg) 

23 

90.9% 

1 13.3% 

Ravleigh  MMAE 

NA 

0.0% 

0.0% 

Beta 

20 

1 5  4°  o 

80  9% 
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CY95  Results  for  All  Development  Data  f 

Score 

MAPL 

SDAPE 

(.'Pi  3 

16 

16  7% 

19  5% 

CPI6 

9 

15.7% 

18  5% 

CPIeum 

12 

16.2% 

18.6% 

SP13 

6 

15  3% 

18  1% 

SPIb 

3 

14  5% 

18.3% 

SPIcum 

3 

14  9% 

18  1% 

SC  13 

18 

17  2% 

20  5% 

SC  1 6 

1  1 

15.3% 

19  0% 

SC' 1  cum 

0 

15.2% 

18  7% 

CP!6  \  SPIcum 

8 

15  1% 

18.7% 

Composite 

Id 

15.8% 

18.5% 

Rayleigh  i Lin  Reg) 

24 

338.4% 

5079.9% 

Ravleigh  iT-G  Reg  i 

->-> 

264  4% 

2424.0% 

Raxleigh  MMAL 

20 

236  1% 

536.9% 

Beta 

26 

210541  1% 

4194158  9% 

CY95  Results  for  Development  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

17 

26.7% 

26.4% 

CP  it' 

13 

25.7% 

25.2% 

CPIeum 

If' 

26.0% 

23.4% 

SPi3 

6 

22  7% 

23  8% 

SP16 

8 

22  0% 

25  9% 

SPIcum 

3 

21  2% 

23.5% 

SC  1 3 

21 

28  0% 

28.4% 

SCI6 

15 

24. 5° o 

26.7% 

SO  cum 

v 

23  4% 

24  3% 

CP!6  \  S  Pi  cum 

12 

22  9% 

26  5% 

Composite 

8 

24.9% 

23  4% 

Raxleigh  i  Lm  Reg) 

25 

1 117% 

88  4% 

Raxleigh  i  T-G  Reg) 

23 

105  6% 

64  1% 

Raxleigh  MMAL 

27 

746  9% 

1001  8% 

Beta 

29 

933891.2% 

8842693  4% 

CY95  Results  for  Development  Data  >50%  to 
75%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

6 

113% 

10  7% 

CPI0 

6 

1 0.6% 

115% 

CPIeum 

9 

1!  1% 

118% 

SP13 

13 

12  4% 

11.9% 

SPi6 

13 

116% 

12.0% 

SPIcum 

/> 

1  1.1% 

1  1 .4% 

SC  13 

17 

12.5% 

12  3% 

SC  16 

1 1 

10  6% 

12  2% 

SCI  cum 

6 

10.4% 

117% 

CPio  \  SPIcum 

4 

10. 1% 

115% 

Composite 

8 

11  1% 

117% 

Ravleigh  (I. in  Regi 

21 

67.7% 

120  3% 

Raxleigh  i T-G  Reg) 

672.3% 

5035  9% 

Raxleigh  MMAL 

23 

104.1% 

126  5% 

Beta 

N 

59  7% 

115  5% 

Best  Performing  Method(s) 

Overall  SPI6  SPIcum 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPIcum 

SP13 

CPI6  x  SPIcum 
SPIcum 


CY95  Results  for  Development  Data  >25%  to 
50%  Complete 

Score 

MAPE 

SDAPE 

CP13 

12 

218% 

26.1% 

CP16 

12 

21  6% 

27.0% 

CPIeum 

14 

213% 

27  9% 

SPI3 

7 

21  1% 

25.9% 

SP16 

10 

21.0% 

27  2% 

SPIcum 

11 

20  7% 

28.1% 

SC13 

9 

21.7% 

25.0% 

SCI6 

10 

213% 

26  9% 

SCIcum 

1  1 

20.0% 

28  4% 

CP16  x  SPIcum 

10 

20.9% 

27.6% 

Composite 

13 

21.1% 

27  9% 

Ravleigh  (Lin  Reg) 

25 

1533.0% 

11511  0% 

Ravleigh  (T-G  Reg) 

22 

266  3% 

764  0% 

Rayleigh  MMAE 

22 

352  8% 

450  6% 

Beta 

27 

2829  1% 

11516  0% 

CY95  Results  for  Development  Data  >75%  to 
100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

14 

1  1  4% 

9.8% 

CP16 

14 

11  7% 

9.3% 

CPIeum 

6 

10  8% 

6.9% 

SP13 

5 

99% 

7.0% 

SP16 

5 

9.9% 

7  0% 

SPIcum 

3 

10.6% 

6.5% 

SCI3 

14 

1 1.1% 

10.8% 

SC  16 

10 

1 1  0% 

9.2% 

SCIcum 

7 

10  9% 

6  9% 

CPI6  x  SPIcum 

15 

1 1  9%' 

9  3% 

Composite 

5 

10  8% 

6.8% 

Ravleigh  (Lin  Reg) 

19 

14  1% 

15.4% 

Ravleigh  (T-G  Reg) 

24 

100.6% 

cr 

oo 

Ravleigh  MMAE 

21 

36  7% 

55.6% 

Beta 

24 

36  8% 

222.0% 
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I  ^  Results  for  All  Production  Data  1 

Score 

MAPE 

SHAPE 

CPI  5 

13 

1 1  6% 

20  0% 

CP16 

6 

10.3% 

11.8% 

C  Pi  cum 

4 

9  8% 

1  1 .4% 

SPI3 

20 

15.1% 

26.5% 

SPlo 

10 

12.5% 

1 5  7% 

SPicum 

1  ! 

12.1% 

17  9% 

SC  13 

21  5% 

124.7% 

SC  16  ' 

17 

14.1% 

19  7% 

SCicum 

14 

13.1% 

19  1% 

CPI6  \  SPicum 

13 

13  3% 

16  9% 

Composite 

2 

9.4% 

10.7% 

Rayleigh  (Lin  Reg) 

1  26 

284.4% 

1605  2% 

Raxieiph  (T-G  Reg) 

24 

264.2% 

835.0% 

Ra\  leiiih  MMAK 

N  A 

0  0% 

0.0% 

Beta 

28 

4037.4% 

79274  1% 

T\  Results  for  Production  Data  0%  to  25% 
Complete 

[L  Score  |  MAPEI  SDAPF.I 

C  PI 5 

15 

21.0% 

34.0% 

CPiO 

6 

1 8  3% 

17.0% 

CPIcum 

4 

172% 

16  4% 

SPJ3 

->  ~i 

28  3% 

44  1% 

SPIo 

Q 

22  7°0 

23  0% 

SPicum 

14 

22  9% 

29  1% 

SC  13 

24 

47  6% 

233  3% 

SC  16 

W 

27  1% 

30  7% 

SCii  urn 

17 

26.0% 

30  5% 

CPI 6  \  SPicum 

13 

25.8% 

25.5% 

Composite 

2 

15.8% 

15  2% 

Ravleigh  [  l.m  Ree  i 

20 

105  1% 

28.5% 

Ra  \  ieigh  i  T-G  Red  > 

17 

101.5% 

24  4% 

Rasleigh  MMAK 

N  A 

0.0% 

o 

o 

Beta 

28 

14549.5% 

152444.3%| 

1  ^  Results  for  Production  Data  >50%  to  75% 
Complete 

[  Score  I  MAPE 

SHAPE 

CPI  3 

6 

8.4% 

8  0% 

CPP’ 

4 

7.7% 

7.9% 

CPIcum 

|  4 

7.5% 

8.1% 

SPI3 

19 

11.2% 

10  3% 

SP16 

14 

9.6% 

8  8% 

SPicum  1 

9 

8.5% 

8.3% 

SC  1 3 

21 

1  1  8% 

1 1  2% 

SC  16 

17 

10.2% 

9.6% 

SCicum 

i  1 

8.7% 

8.7% 

CPI6  \  SPicum 

14 

9.0% 

9.3% 

Composite 

s 

7  6% 

8.1% 

Ravleigh  i  Lin  Reo ) 

26 

469  4% 

1912  9% 

Ra> lei uh  (T-G  Reg i 

26 

476.8% 

1410.9% 

Rayleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

23 

47  1% 

62.9% 

Best  Performing  Method(s) 

Overall  Composite 


0°  o  -  25%  Composite 

>25%  -  50%  Composite 

>50%  -  75%  CP16  CPIcum 


>75%- 100%  CPIcum  Composite 


TY  Results  for  Production  Data  >25%  to  50% 
Complete 

I  Score  |  MAPEl  SHAPE  1 

CPI3 

7 

10.7% 

10.6% 

CP16 

4 

104% 

9.9% 

CPIcum 

4 

10.2% 

r  io.o% 

SPI3 

19 

16  9% 

15  6% 

SPI6 

15 

15  4% 

14  3% 

SPicum 

9 

13  4% 

i  1.4% 

SCI3 

21 

19  5% 

20  6% 

SCI6 

17 

16  4% 

15  5% 

SCicum 

1  1 

13.5% 

12.0% 

CPI 6  x  SPicum 

13 

14.1% 

12.5% 

Composite 

3 

10.1% 

10.0% 

Rayleigh  (Lin  Reg) 

23 

165.5% 

296.2% 

Ravleigh  (T-G  Reg) 

25 

234.1% 

874  9% 

Ravleiph  MMAE 

N  A 

0.0% 

0.0% 

Beta  ! 

27 

529  8% 

2852.2% 

T\  Results  for  Production  Data  >75%  to  100% 
Complete 

Score 

MAPI: 

SHAPE 

CPI  3 

12 

6.0% 

4.8% 

CPI6 

10 

5.8% 

4.7% 

CPIcum 

3 

5.1% 

3.1% 

SPI3 

6 

5.1% 

4  0% 

SPI6 

4 

4  8% 

3  6% 

SPicum 

5 

5.2% 

3.4% 

SCI3 

15 

6  1% 

6.4% 

SCI6 

11 

5.4% 

5.1% 

SCicum 

6 

5.4% 

3.4% 

CPI6  x  SPJcum 

15 

6.2% 

5.1% 

Composite 

3 

5  1% 

3.1% 

Ravleigh  (Lin  Reg) 

2° 

367.9% 

2256  9% 

Ravleigh  (T-G  Reg) 

20 

231.8% 

450.8% 

Rayleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

18 

126% 

27  1% 
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f  C  V 95  Results  for  All  Production  Data  1 

Score 

MAPE 

SDAPE 

CPI  5 

10 

11.0% 

19  9% 

C'Pl6 

6 

9.6% 

11.4% 

CPIcum 

4 

9  ]  % 

1 1.0% 

SP13 

20 

15  3% 

28  3% 

SPI6 

10 

12.4% 

17.1% 

SPIcurn 

12 

12.2% 

20.5% 

SC  13 

22 

21.5% 

121.2% 

SC  16 

17 

14  1% 

211% 

SC  i  cum 

16 

13.2% 

218% 

CPI 6  x  SPIcurn 

13 

13  3% 

18.5% 

Composite 

8  7% 

10  3% 

Ravleigh  (Lin  Reg) 

25 

197.5% 

623  8% 

Rayleigh  f T-G  Rep) 

28 

326  5% 

2592  6% 

Rayleigh  MMAF 

25 

209.6% 

360.1% 

Beta  1 

30 

3828  9% 

86582  3% 

C  Y95  Results  for  Production  Data  0%  to  25%  I 

_ 

Complete 

1  ..  II  Score 

1  MAPE'I  SDAPFl 

CPI  3 

12 

20.1% 

33  9% 

CPI6 

6 

1  7  0% 

16.6% 

CPIcum 

4 

16.0% 

15  7% 

SPi3 

21 

29  9% 

47  4% 

SPI6 

9 

23  8% 

25.7% 

SPIcurn 

15 

24  2% 

34.1% 

SC  13 

24 

49.0% 

226  4% 

SCI  6 

16 

28.3% 

33.3% 

SCI  cum 

18 

27  4% 

35  7% 

CPI6  x  SPIcurn 

12 

26.9% 

28  8% 

Composite 

2 

14.6% 

14  2% 

Rayleigh  (Lin  Res) 

2  ^ 

109  6% 

55  7% 

Rayleigh  iT-G  Reg)  | 

18 

101.6% 

29.1% 

Rayleigh  M MAE 

:  28 

295  7% 

513.2% 

Beta  j 

30 

13764.6% 

166637.2% 

( 95  Results  for  Production  Data  >50%  to  75% 
Complete 

!  Score 

MAPE 

SDAPE 

CP!  3 

4 

8.2% 

8  0% 

CPI  6 

3 

7.6% 

8.0% 

CPIcum 

3 

7.5% 

8.2% 

SPI3 

17 

10.8% 

9  8% 

SPI6 

12 

9.2% 

8  8% 

SPIcurn 

7 

8.5% 

8.3% 

SC  13 

19 

1 1  0% 

1 1.2% 

SC16  ^ 

15 

9  8% 

9  7% 

SCIcum 

9 

8  6% 

8  7% 

CPI 6  x  SPIcurn 

12 

8.9% 

9.3% 

Composite 

3 

7  5% 

oo 

vo 

Rayleigh  ( Lin  Reg) 

26 

224.6% 

637  0% 

Rayleigh  (T-G  Reg) 

26 

233.2% 

439  3% 

Rayleigh  MMAE 

23 

207.0% 

250  3% 

Beta 

21  "" 

46  9% 

61  6% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


Composite 

CPI6  Composite 

CPI6  CPIcum  Composite 

CPIcum 


CY95  Results  for  Production  Data  >25%  to  50% 
Complete 

|  Score 

MAPE 

SDAPE 

CPI  3 

7 

10.6% 

10.0% 

CP16 

3 

10.0% 

9.3% 

CPIcum 

4 

9  8% 

9.5% 

SPI3 

19 

16  9% 

15  8% 

SPI6 

15 

15.3% 

14  4% 

SPIcurn 

9 

13.3% 

!  1.5% 

SCI3  “ 

2! 

19  5% 

n  |  to- 

SCI6  ~~1 

17 

16  5% 

1 5.6% 

SCicum 

1  1 

13  7% 

12.1% 

CPI6  x  SPIcurn 

13 

14  2% 

12  7% 

Composite 

3 

9  8% 

9.4% 

Ravleigh  (Lm  Reg) 

26 

206.3% 

687.3% 

Rayleigh  (T-G  Reg) 

23 

153.0% 

194.8% 

Ravleigh  MMAE 

1  26 

312  1% 

451.5% 

Beta  | 

29 

546  2% 

2917.8% 

CY95  Results  for  Production  Data  >75%  to  100% 
Complete 

||  Score 

MAPE 

SDAPE 

CPI3 

1  >4 

5.2% 

5.2% 

CPI6 

I  1 

5.0% 

4  6% 

CPIcum 

2 

4.3% 

3  7% 

SPI3 

7 

4.6% 

4  4% 

SPI6 

3 

4.3% 

3  8% 

SPIcurn 

4 

4.4% 

3.8% 

SCI3 

16 

5.3% 

6.2% 

SCI6 

9 

4.8% 

4.5% 

SCicum 

6 

4.6% 

3.9% 

CPI6  x  SPIcurn 

13 

5  2% 

4  8% 

Composite 

3 

4  4% 

3.7% 

Rayleigh  (Lm  Reg) 

~)  -> 

232.1% 

753  7% 

Rayleigh  (T-G  Reg) 

24 

572.9% 

41 19.9% 

Ravleigh  MMAE 

20 

109.2% 

177.5% 

Beta 

18 

1 2  4% 

26  9% 
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f~~  TV  Results  for  All  Other  Data  | 

Score 

MAPE 

SDAPE 

CPI  3 

13 

12.4% 

13  4% 

CPI  6 

5 

10  8% 

12.6% 

CPIcum 

!  1 1 

1 1  6% 

13.1% 

SPi.3 

15 

13  0% 

1 4  4% 

SP16 

7 

3 1.2% 

12  6% 

SPlcum 

8 

1  1  4% 

12.6% 

SC  1 3 

17 

15  4% 

16.8% 

SCI  6 

5 

1  11% 

12.1% 

SClcum 

3 

10  4% 

12.1% 

CPI6  \  SPlcum 

3 

10.8% 

1 1.4% 

Composite 

9 

11.4% 

12.7% 

Ray  lei. eh  ( Lin  Reg  i 

19 

133  1% 

260  4% 

Ravleieh  iT-G  Ree) 

23 

2355.4% 

18148.8% 

Rayleigh  MMAE 

N  A 

0.0°  o 

00% 

Beta 

21 

3  573.3% 

31203  0% 

Best  Performing  Method(s) 

Overall  SClcum  CPI6  x  SPlcum 

0%  -  25%  SC  16 


>25% -50%  CP16  CPIcum  SClcum 

>50%  -  75%  CPIcum 

>75%-  100%  Beta 


TV  Resuits  for  Other  Data  0%  to  25%  Complete 


i  Score  |  MAPEl  SDAPeI 

CPI  3 

15 

23  1% 

1 8  0% 

C  P 1 6 

15 

22  0% 

19  3% 

CPIcum 

15 

23.3% 

17.6% 

SI  >13  H 

1  1 

23.1% 

17  2% 

SPI6 

4 

20.4% 

1  5  8% 

SPlcum 

6 

20  4% 

16  5% 

SC  13 

14 

26  2% 

17  3% 

SCh 

3 

19  4% 

16  3% 

SClcum 

6 

1 9  6% 

16  8% 

CPi(>  \  SPlcum 

6 

19  4% 

16.9% 

Composite 

12 

22  0% 

1  7  4% 

Ravleigh  i  lan  Ree) 

i 

95  6% 

24  6% 

Ra\leigh  i T-G  Ree) 

!  31 

95.3% 

37  1% 

Rayleigh  MMAE  j 

N  A 

0.0% 

0.0% 

Beta  | 

24 

5768.8% 

21503.2% 

TV  Results  for  Other  Data  >50%  to  75% 
_ Complete 


Score 

MAPE 

SDAPE 

CP13 

7 

8  1% 

7.2% 

CPJo 

10 

8  1% 

76% 

CPIcum 

7.1% 

7.1% 

SPI3 

15 

8.9% 

10.1% 

SPI6 

12 

8.0% 

90% 

SPlcum 

1  1 

7  6% 

90% 

SC  1 3 

19 

1 1  4% 

12  2% 

SCi6 

14 

9.4% 

8  8% 

SClcum 

6 

7.3% 

7  5% 

CPIS  \  SPlcum 

14 

9.5% 

8.2% 

Composite 

5 

7.3% 

7.4% 

Rayleigh  ( Lin  Reg) 

23 

1  19.2% 

175.3% 

Rayleigh  i  f -G  Reg) 

25 

243.7% 

469  8% 

Ra\  leieh  MMAE 

N  A 

0  0% 

0  0% 

Beta 

21 

19.2% 

25.3% 

TV  Results  for  Other  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

12 

13  6% 

12.6% 

CP16 

4 

10  9% 

10.5% 

CPIcum 

4 

1 1.0% 

10.1% 

SP13 

18 

16.3% 

15.5% 

SPI6 

16 

15.6% 

14.3% 

SPlcum 

10 

13  2% 

1 1  0% 

SCI  3 

20 

18  5% 

1  7.2% 

SC  16 

14 

14  0% 

13  3% 

SClcum 

4 

9.0% 

1 0  7% 

CP16  x  SPlcum 

7 

11.9% 

10.5% 

Composite 

5 

114% 

10  1% 

Rayleigh  (Lin  Reg) 

22 

244  5% 

479.9% 

Rayleigh  (T-G  Reg) 

26 

656  0% 

2486  2% 

Rayleigh  MMAE 

N.  A 

0  0% 

0  0% 

Beta 

24 

254.9%' 

589  5% 

TV  Results  for  Other  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

17 

6.2% 

6.2% 

CP16 

9 

5.6% 

5.3% 

CPIcum 

6 

5.5% 

5.0% 

SPI3 

13 

5  4% 

6.9% 

SPI6 

10 

5.1% 

6.0% 

SPlcum 

10 

5.5% 

oo 

SC13 

20 

7.3% 

13.9% 

SC  1 6 

10 

5.4% 

5.9% 

SClcum 

9 

5.7% 

5.2% 

CP16  x  SPlcum 

12 

5.9% 

5  5% 

Composite 

7 

5.5% 

5.2% 

Rayleigh  ( Lin  Reg) 

22 

93  8% 

153.4% 

Rayleigh  (T-G  Reg) 

24 

5554  0% 

29187.6% 

Rayleigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

2 

3  4% 

4  8% 
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CY 95  Results  for  All  Other  Data  f 

Score 

MAPE 

SDAPE 

CPI  3 

15 

11.9% 

12  6% 

CP16 

3 

10.4% 

115% 

C  Pi  cum 

1  1 

11.3% 

12  2% 

SP13 

18 

13  6% 

14.2% 

SPk< 

10 

117% 

11.8% 

SPlcum 

15 

12.0% 

12.3% 

SC  13 

20 

14.9% 

1  5  9% 

SC16 

6 

10.7% 

115% 

SCIcum 

6 

10.5% 

119% 

CPi6  x  SPlcum 

4 

10  6% 

10.9% 

Composite 

0 

11.1% 

1 1 .9% 

Ravleigh  ( Lin  Reg) 

on 

1  19  7% 

203.1% 

Ravieigh  iT-G  Reg! 

24 

283.3% 

738  1% 

Ravleigh  MMAL 

26 

377.6% 

851  1% 

Beta 

28 

3287.0% 

24172  5% 

CY95  Results  for  Other  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

14 

22.4% 

17  1% 

CP  16 

12 

19.9% 

17.8% 

CPlcum 

14 

21.1% 

17  4% 

SP13 

17 

24.1% 

17  8% 

SP16 

7 

20  4°  o 

14  0% 

SPlcum 

10 

21.1% 

16.5% 

SC  13 

w 

27.7% 

19  3% 

SC  16 

4 

18  7% 

16  0% 

SCIcum 

8 

196% 

16  9% 

CPI6  \  SPlcum 

S 

18  6% 

16  7% 

Composite 

;  i 

20  0%, 

17.1% 

Ravleigh  (I  in  Reg) 

n  -s 

98  1% 

27.5% 

Ravleigh  (T-G  Reg) 

22 

97.0% 

41.9% 

Ravleigh  MM AE 

25 

688.4% 

960.1% 

Beta 

o? 

12279.8% 

46424.3% 

CY 95  Results  for  Other  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPU 

4 

7  8% 

6.4% 

CPI6 

6 

7  9% 

6  8% 

CPlcum 

6 

oc 

6.1% 

SP13 

19 

9.9% 

8.9% 

SP16 

16 

9. 1% 

8.0% 

SPlcum 

14 

8.8% 

7.8% 

SC  13 

IV 

9  7% 

9  9% 

SC  16 

10 

8  1% 

7.5% 

SCIcum 

6.9% 

69% 

CP16  \  SPlcum 

10 

8.2% 

7.0% 

Composite 

8 

8  3% 

6  4% 

Ravleigh  (Lin  Reg) 

24 

1 13.6% 

155  3% 

Ravleigh  (T-G  Reg) 

28 

445.0% 

1132.2% 

Ravleigh  MMAE 

26 

330.5% 

682.6% 

Beta 

"O 

17.5% 

22  0% 

Best  Performing  Method(s) 


Overall  CPI6 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%  -  1 00% 


SC  16 

CPI6  CPlcum  SCIcum 

CPI  3 

Beta 


CY95  Results  for  Other  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

12 

12.6% 

12  0% 

CPI6 

5 

10.2% 

10.2% 

CPlcum 

5 

10.0% 

10.3% 

SPI3 

19 

16  2% 

15.1% 

SP16 

17 

15  4% 

14  4% 

SPlcum 

12 

13  0% 

11.4% 

SC  13 

21 

17  8% 

16.5% 

SC  16 

15 

13.6% 

12.9% 

SCIcum 

5 

9.3% 

10  4% 

CP16  x  SPlcum 

6 

12  1% 

10  1% 

Composite 

8 

10  4% 

10  4% 

Ravleigh  {Em  Reg) 

23 

167  9% 

284  5% 

Ravleigh  (T-G  Reg) 

27 

295  1% 

664  7% 

Ravleigh  MMAE 

29 

578  9% 

1354.8% 

Beta 

25 

258  7% 

610.3% 

CY95  Results  for  Othe 
Com 

r  Data  >75%  to  100% 
plete 

Score 

MAPE 

SDAPE 

CPI3 

6 

62% 

5.0% 

CP16 

8 

6.3% 

5.1% 

CPlcum 

5 

6.2% 

4.7% 

SPI3 

7 

6.0% 

5.2% 

SPI6 

8 

5.9% 

5.4% 

SPlcum 

7 

6.2% 

5  1% 

SC  13 

11 

62% 

5  8% 

SC  16 

8 

6.0% 

5.3% 

SCIcum 

5 

6.2% 

4  7% 

CPI6  x  SPlcum 

10  ! 

6  4% 

5.2% 

Composite 

5 

6.2% 

4.7% 

Ravleigh  (Lin  Reg) 

16 

105  8% 

227  8% 

Rayleigh  (T-G  Reg) 

19 

233.2% 

559.0% 

Ravleigh  MMAE 

16 

108  9% 

1 19.2% 

Beta 

-> 

3.1% 

4  7% 
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|  TV  Results  for  All  Cost  Plus  Data  j 

Score 

MAPE 

SDAPE 

CPI  3 

16 

13.2% 

1 5  6% 

CP16 

6 

1  1 .2% 

13.6% 

CPI  cum 

13 

12  0% 

14  5% 

SPI3 

13 

12  8% 

14.3% 

SP16 

5 

11.1% 

13.6% 

SPIcum 

9 

12.0% 

13  7% 

SC  1 3 

18 

13  9% 

16  6% 

SC  16 

4 

10  9% 

13  6% 

SCIcum 

7 

1  1  1% 

13.8% 

CP  16  \  SPIcum 

10  8% 

13  0% 

Composite 

10 

118% 

14.2% 

Rayleigh  (Lin  Reg) 

20 

108.0% 

368.4% 

Rayleigh  (T-G  Reg) 

22 

205  4% 

656.9% 

Ra\leigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

24 

6510.5% 

60060.2% 

JX  Results  for  Cost  Plus  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

20 

28.3% 

20.0% 

CPI6 

13 

26.2% 

16.9% 

CPIcuin 

14 

26  9% 

16.6% 

SP13 

10 

27  1% 

15.3% 

SPI0 

8 

26  3% 

15.3% 

SPIcum 

6 

24  6°  o 

15.9% 

SCI  3 

20 

29  9% 

19.6% 

SC  16 

6 

24  5% 

16.2% 

SCIcum 

10 

23  0% 

17.2% 

CP16  x  SPIcum 

6 

22  8% 

16.5% 

Composite 

9 

25.8% 

16.4% 

Rayleigh  < Lin  Reg) 

23 

111.0% 

65.1% 

Ravleigh  (T-G  Reg) 

25 

164  9% 

235  4% 

Ra\!eigh  MM  At 

N  A 

0  0% 

0  0% 

Beta 

27 

29043.1% 

1 2984 1  1  % 

TX  Results  for  Cost  Plus  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPU 

17 

8.6% 

6.8% 

CPI  6 

9 

7.4% 

5.9% 

CPIcuin 

6 

7  1% 

5  8% 

SPI3 

14 

7.9% 

6.1% 

SPI6 

12 

7.4% 

6.5% 

SPIcum 

12 

7.8% 

6.0% 

SC  13 

19 

8  7% 

8.3% 

SC!  6 

8 

6.9% 

6.5% 

SCIcum 

3 

7.0% 

5.5% 

CPI6  x  SPIcum 

8 

7.5% 

5.7% 

Composite 

7 

7.2% 

5.8% 

Ra\ leigh  i  Ltn  Reg) 

23  ! 

183  8% 

515.2% 

Ra\leigh  iT-G  Reg) 

25  ! 

295.3% 

695.6% 

Ravleigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

21 

64  9% 

112  7% 

Best  Performing  Method(s) 

Overall  CPI6  x  SPlcuin 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPlcum  SC16  CPI6  x  SPIcum 
SPIcum  CP16x  SPIcum 
SCIcum 
CPlcum 


TV  Results  for  Cost  Plus  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

11 

20  5% 

16  8% 

CP16 

8 

19.8% 

16  8% 

CPlcum 

9 

18  8% 

17  3% 

SP13 

10 

19.9% 

16  9% 

SP16 

13 

20  7% 

16  9% 

SPIcum 

7 

20  2% 

16.2% 

SC13 

9 

20.2% 

16.7% 

SCI6 

1  1 

20  7% 

16.7% 

SCIcum 

9 

18.0% 

17.4% 

CPI6  x  SPIcum 

7 

199% 

16  4% 

Composite 

9 

18.9% 

17.2% 

Rayleigh  (Lin  Reg) 

19 

178.7% 

595  2% 

Rayleigh  (T-G  Reg) 

19 

192  4% 

468  0% 

Rayleigh  MMAE 

N/A 

0.0% 

0.0% 

Beta 

22 

24884% 

8981  7% 

TX  Results  for  Cost  Pit 

Com 

IS  Data  >75%  to  100% 
plete 

Score 

MAPE 

SDAPE 

CPI  3 

12 

4.5% 

4  9% 

CP16 

14 

4  4% 

5.2% 

CPlcum 

3 

3  7% 

3  4% 

SP13 

1 1 

4.3% 

5  0% 

SP16 

4 

3.6% 

3.6% 

SPIcum 

7 

3  9% 

3.7% 

SCI3 

19 

4  9% 

8  8% 

SCI6 

11 

4  0% 

5.1% 

SCIcum 

5 

39% 

3  5% 

CP16  x  SPIcum 

17 

4.6% 

5.4% 

Composite 

4 

3.7% 

3.5% 

Ravleigh  (Lin  Reg) 

->3 

34.8% 

98  7% 

Ravleigh  (T-G  Reg) 

25 

176  9% 

775.1% 

Rayleigh  MMAE. 

N  A 

0.0% 

0  0% 

Beta 

21 

20.3% 

89  4% 
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j  CY95  Results  for  All  Cost  Plus  Data  \ 

Score 

MAPE 

SDAPE 

CPI  3 

14 

14.6% 

15.5% 

CPI  6 

10 

13  4% 

13.3% 

CPI  cum 

1  1 

13.5% 

13.3% 

SP 13 

1  I 

13  8% 

13.2% 

SPlo 

12  7% 

12.7% 

SPIcum 

6 

13.4% 

12.7% 

SC13 

10 

15.5% 

16.2% 

SC  It' 

8 

13  3% 

13.2% 

SCIcum 

4 

13  1% 

1 2.8% 

CPI 6  \  SPIcum 

0 

1 3.4% 

12.7% 

Composite 

6 

13.2% 

13  0% 

Ra\  lei  eh  (Lin  Reg) 

22 

41 9.7% 

5649.4% 

RaCeigh  ('i  -G  Reel 

20 

325.6°  o 

2715.1% 

Ravleigh  MMAE 

18 

260.8% 

657  6% 

Beta 

24 

5380.7% 

34021  8% 

Best  Performing  Method(s) 

Overall  SP16 

0%  -  25%  CPI6  x  SPIcum 


>25%  -50%  SPIcum 

>50% -75%  SCIcum 

>75%- 100%  SPI3 


CY95  Results  for  Cost  Plus  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

20 

23.7% 

23  0% 

CPI  6 

13 

21.0% 

18  3% 

CPI  cum 

1  5 

21  5% 

17.8% 

SPI3 

14 

22  5% 

17  1% 

SPI6 

8 

21.1% 

16  7% 

SPIcum 

7 

20  1% 

1 7.0% 

SC  13 

20 

27.7% 

22  7% 

SCI  6 

8 

2 1 .3% 

16  4% 

SCIcum 

6 

20. 1% 

16.8% 

(.  Pit*  \  Si'icum 

2 

19  7% 

16.0% 

Composite 

lo 

20  4% 

17  2% 

Ra> leigh  <  Lin  Reg t 

^  ; 

109.6% 

77  3% 

Rayleigh  i T-G  Reg) 

25 

121.7% 

77.7% 

RaCetgh  MMAE 

7 

781  1% 

1062  7% 

Beta 

29 

22353  3% 

71408  9% 

CY95  Results  for  Cost  I 

Com 

*lus  Data  >50%  to  75% 
plete 

Score 

MAPE 

SDAPE 

CPI  3 

I  ^ 

8  4% 

6  2% 

.CP  16 

8 

7.8% 

6  0% 

CPI  cum 

7 

7  8% 

5.9% 

SP13 

12 

8.6% 

5  9°  o 

SPlo 

12 

8.1% 

6.3% 

SPIcum 

10 

8.5% 

5  8% 

SC  1 3 

15 

8.5% 

7.1% 

SC  1 6 

7 

7  '1% 

6.3% 

SCIcum 

3 

7.3% 

5  7% 

C PIG  x  SPIcum 

<; 

7  6% 

5  8% 

Composite 

8 

7  9% 

5  9% 

Ravleigh  (Lin  Reg) 

20 

92.0% 

169.9% 

Rayleigh  i  T-G  Reg) 

24 

950.8°  o 

5651.8% 

Ravleigh  MMAE 

22 

152.7% 

186.3% 

Beta 

18 

62  3°  o 

109  6% 

CY95  Results  for  Cost  Plus  Data  >25%  to  50% 
Complete 

|  [|  Score 

MAPE 

SDAPE 

CPI3 

11 

192% 

16  9% 

CP16 

8 

18  7% 

16.8% 

CPIcum 

9 

1 7.3% 

17.5% 

SPI3 

9 

18  4% 

17  2% 

SPI6 

13 

19  1% 

17  4% 

SPIcum 

7 

18.5% 

16  8% 

SC  13 

10 

19.4% 

16  6% 

SCI6 

14 

20. 1% 

17.0% 

SCIcum 

8 

1 7  4°  o 

17.2% 

CPI6  x  SPIcum 

0 

19  4% 

16  4% 

Composite 

9 

17.3% 

17.5% 

Ravleigh  (Lin  Reg) 

23 

1729  8% 

12478  5% 

Ravleigh  (T-G  Reg) 

19 

220  2% 

771.1% 

Ravleigh  MMAE 

21 

434.5% 

955.1% 

Beta 

25 

3776  9% 

12851.9% 

CY 95  Results  for  Cost  Plus  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

16 

1 1.0% 

10.3% 

CPI  6 

16 

11.3% 

10  0% 

CPIcum 

8 

10.3% 

7.5% 

SPI3 

3 

9  7% 

6.8% 

SPI6 

4 

9.3% 

7  4% 

SPIcum 

5 

10.1% 

7.0% 

SCI3 

15 

10  8% 

10.3% 

SCI6 

10 

10.4% 

9  6% 

SCIcum 

10 

10.5% 

7.5% 

CP16  x  SPIcum 

16 

1 1.4% 

9.9% 

Composite 

7 

10.3% 

7.4% 

Ravleigh  (Lin  Reg) 

25 

70.4% 

540.3% 

Ravleigh  (T-G  Reg) 

24 

1 1 3  4% 

146  6°  o 

Ravleigh  MMAE 

21 

48.9% 

80  9°  o 

Beta 

44  7% 

240  6% 
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|  TV  Results  for  All  Fixed  Price  Data 

Score 

MAPE 

SDAPE 

CPI  3 

12 

12  7% 

20  7% 

CPI  6 

4 

11.4% 

15  0% 

CPIcum 

5 

1 1 .4% 

15  2% 

SPI3 

17 

15  4% 

25.7% 

SP16 

8 

13.0% 

17  5% 

SPIcum 

9 

12  7% 

19  1% 

SC  13 

19 

20  9% 

110  7% 

SC  16 

14 

14  4% 

20.5% 

SC  [cum 

12 

13.6% 

20.2% 

CPI6  x  SPIcum 

10 

13  6% 

18  4% 

Composite 

1  10% 

14.7% 

Rayleigh  (Lin  Ret;) 

21 

243  0% 

1426  6% 

Rayleigh  (T-G  Reg) 

23  ' 

550  1% 

6940  3% 

Ravleigh  MM  AH 

N  A 

0.0% 

0.0% 

Beta 

25 

85417  6% 

2470694  6% 

TV  Results  for  Fixed  Price  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

14 

23.0% 

32.7% 

CP16 

6 

20.8% 

20.2% 

CPIcum 

4 

20.6% 

19  8% 

SP!3 

21 

27.7% 

41.0% 

SPI6 

9 

23.1% 

24  1% 

SPIcum 

13 

23  3% 

28.6% 

SC  1 3 

24 

43  7% 

206  6/o 

SC  16 

18 

27.2% 

30  4% 

SClcum 

16  : 

26  5% 

29.8% 

CP16  \  SPIcum 

13  ! 

26  1% 

26  3% 

Composite 

2 

19  3% 

19  1% 

Rayleigh  1 1. in  Reg) 

25  ; 

105.8% 

53.5% 

Ravleigh  iT-C  Reg) 

17 

97.2% 

24  7% 

Ravleigh  MM  AH 

N  A 

0  0% 

0.0% 

Beta 

28 

312849.5% 

4729493.2% 

TV  Results  for  Fixed  Price  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

4 

9  5% 

9.8% 

CPI6 

3 

8  8% 

10.1% 

CPIcum 

4 

8.8% 

10.5% 

SPI3 

16 

12  4% 

12  1% 

SPIo 

12 

10.8% 

1 1  0% 

SPIcum 

6 

9.3% 

10.6% 

SC  13 

18 

13.4% 

13  0% 

SC  1 6 

14 

1 1.2% 

11.4% 

SClcum 

0 

9  6°  o 

10.8% 

CP16  x  SPIcum 

1 1 

9.8% 

1 1 .0% 

Composite 

4 

8.8% 

10.5% 

Rayleigh  ( l.in  Reg) 

23 

358.8% 

1662.5% 

Ra> leigh  (T-G  Reg) 

403  4°  o 

1249  0% 

Ravleigh  MMAE 

N  A 

0  0% 

0.0% 

Beta 

42.1% 

68.3% 

Best  Performing  Method(s) 

Overall  Composite 

0°  o  -  25%  Composite 

>25% -50%  CPI6 

>50%  -  75%  CPI6 


>75%- 100%  CPIcum  Composite 


TV  Results  for  Fixed  Price  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

4 

12.9% 

17  4% 

CPI6 

3 

12  4% 

17.5% 

CPIcum 

5 

12.5% 

1 8  0% 

SPI3 

17 

1 8.2% 

19  8% 

SP16 

14 

16  6% 

19  6% 

SPIcum 

9 

14  7% 

186% 

SC  13 

20 

20  5% 

23.1% 

SC  16 

17 

17  0% 

20.1% 

SClcum 

10 

14  3% 

190% 

CP16  x  SPIcum 

1  ! 

14.9% 

18.8% 

Composite 

4 

12  4% 

18  0% 

Ravleigh  (Lin  Reg) 

193.2% 

4563% 

Ravleigh  (T-G  Reg) 

26 

352.7% 

1323.5% 

Ravleigh  MMAE 

N-A 

0.0% 

0.0% 

Beta 

24 

280.7% 

996  0% 

TV  Results  for  Fixed  Price  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

12 

6.0% 

5  0% 

CP  1 6 

8 

5  6% 

4.2% 

CPIcum 

3 

5.1% 

34% 

SPI3 

6 

5.1% 

4.2% 

SP16 

4 

4  9% 

3  9% 

SPIcum 

4 

5  1% 

3.7% 

SCI3 

14 

6.3% 

6.8% 

SCI6 

8 

5.4% 

4.5% 

SClcum 

5 

5.4% 

3.7% 

CP16  x  SPIcum 

1  1 

6.0% 

4.6% 

Composite 

3 

3.4% 

Rayleigh  (Lin  Reg) 

18 

293.7% 

1 996.2°  o 

Rayleigh  (T-G  Reg) 

20 

935  1% 

11015.5% 

Rayleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

16 

9  9% 

21  3% 
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[  CY95  Results  for  All  Fixed  Price  Data  j 

Score 

MAPE 

SDAPE 

CPI  3 

10 

12.4% 

20.5% 

CPI  6 

5 

1 1  0% 

14  8% 

CPIcum 

5 

10  9% 

14  9% 

SPI3 

20 

15.7% 

27.2% 

SPI6 

10 

13.2% 

18.5% 

SPIcum 

12 

12.9% 

21.1% 

SC  13 

22 

210% 

107  7% 

SC  16 

17 

14.5% 

21  7% 

SCI  cum 

17 

13  8% 

22  2% 

CPI6  x  SPIcum 

12 

13.7% 

19  7% 

Composite 

2 

10.6% 

14  4% 

Rax leigh  (Lin  Reg) 

25 

159.3% 

439.9% 

Rax leigh  (T-G  Reg) 

28 

294.5°  o 

2295  8% 

Rax  leigh  MMAF. 

25  ! 

227.3% 

418.3% 

Beta 

30  1 

97240.0% 

2829658  4% 

Best  Performing  Method(s) 


Overall 
0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


Composite 
Composite 
CP13  CPI6 
CPI6 

CPIcum  Composite 


O  95  Results  for  Fixed  Price  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

1  1 

21  9% 

32  5% 

CP16 

6 

19  2% 

19.9% 

CPIcum 

4 

1 8.9% 

19.3% 

SPI3 

21 

28.8% 

43.8% 

SP16 

9 

23  6% 

26  3% 

SPIcum 

14 

24.0% 

32.7% 

SC  1 3 

24 

44.7% 

2006% 

SC  1 6 

18 

27  7% 

32  8% 

SC  I  cum 

18 

27.1% 

34.2% 

CPl6  \  SPIcum 

13 

26  5% 

29  2% 

Composite 

'i 

1 7.6% 

18.5% 

Rax  leigh  (Lin  Reg) 

25 

108  7°  o 

61  9% 

Rax  ieigh  (T-G  Reg) 

17 

96.8% 

26  5% 

Rax  leigh  MMAF. 

28 

3789% 

639  5% 

Beta 

30 

357209.9% 

5424614.9% 

CY95  Results  for  Fixed  Price  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

4 

9.4% 

9  8% 

CPI6 

3 

8.9% 

10.2% 

C  PI  cum 

6 

9.1% 

10.6% 

SP13 

16 

12.2% 

1 1.7% 

SPI6 

12 

10.7% 

1 1 .0% 

SPIcum 

8 

9.7% 

10.5% 

SC  13 

18 

12.5% 

12  6% 

SC  16 

14 

10  8% 

1 1 .5% 

SClcum 

9 

9.6% 

10  8% 

CP16  \  SPIcum 

11 

9.7% 

1 1 .0% 

Composite 

9  1% 

10.6% 

Rax  leigh  (Lin  Reg) 

24 

186.5%' 

558.9% 

Rax  leigh  tT-G  Reg) 

24 

205.1% 

389.7% 

Ravleigh  MMAE 

24 

205.5% 

357.4% 

Beta 

20 

42.6% 

68.9% 

CY95  Results  for  Fixed  Price  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

3 

12.5% 

17.6% 

CP16 

3 

11.8% 

17  8% 

CPIcum 

5 

11.8% 

18  4% 

SPI3 

17 

18.1% 

204% 

SPI6 

14 

16  4% 

20.2% 

SPIcum 

9 

14  5°o 

19.2% 

SC  13 

20 

20  3% 

23.8% 

SCI6 

17 

16.9% 

20.7% 

SClcum 

10 

14.3% 

19.6% 

CPI6  x  SPIcum 

1  1 

14.9% 

19.5% 

Composite 

4 

11.8% 

18  3% 

Rayleigh  (Lin  Reg) 

22 

165  9% 

222.6% 

Ravleigh  (T-G  Reg) 

24 

198  7% 

3686% 

Ravleigh  MMAE. 

27 

3289% 

433.5% 

Beta 

27 

293  4% 

1011  3% 

1  CY95  Results  for  Fixed  Price  Data  >75%  to  I 

j  100%  Complete 

Score 

MAPE 

SDAPE 

CPI3 

14 

6  1% 

5  8% 

CPI6 

10 

5.9% 

5.2% 

CPIcum 

3 

5.3% 

4.4% 

SPI3 

7 

5.4% 

5.2% 

SPI6 

4 

5.2% 

4  6% 

SPIcum 

4 

5.3% 

4.5% 

SCI3 

16 

6.2% 

7.4% 

SCI6 

10 

5.7% 

5.3% 

SClcum 

6 

5.5% 

4  5% 

CPI 6  x  SPIcum 

14 

6.2% 

5.4% 

Composite 

3 

5.3% 

44% 

Ravleigh  (Lin  Reg) 

22 

171.7% 

575.5% 

Ravleigh  (T-G  Reg) 

24 

482.4% 

3645.3% 

Ravleigh  MMAE 

20 

99  0% 

163.6% 

Beta 

18  1 

9.6% 

21.1% 
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|  TY  Results  for  Alt  Mixed  Data  | 

Score 

MAPE 

SDAPE 

CPI  3 

16 

13.9% 

5.9% 

CPI  6 

16 

14  1% 

5  5% 

CPlcum 

15 

14  4% 

4  4% 

SP13 

6 

12.1% 

3  8% 

SPI6 

5 

12.3% 

3  6% 

SPIcum 

5 

12.5% 

3  1% 

SC  1 3 

12 

117% 

6.5% 

SC16 

1  1 

12  1% 

5.6% 

SCicum 

8 

12.6% 

3.7% 

CP10  x  SPIcum 

10 

12.3% 

4  8% 

Composite 

12 

14.0% 

3  9% 

Rayleigh  (Lin  Reg) 

26 

296  6% 

773.2% 

Rayleigh  (T-G  Reg) 

22 

88.8% 

98  8°  o 

Raxleigh  MMAL 

N  A 

0.0% 

0.0% 

Beta 

24 

155.3% 

445  6% 

TY  Results  for  Mixed  Data  0%  to  25%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

17 

^  17.1% 

4.8% 

CPI6 

16 

17.5% 

4  2% 

CPlcum 

1  I 

16.5% 

3.3% 

SPI3 

0 

9.4% 

4.8% 

SPI6 

9 

10.1% 

4.5% 

SPIcum 

5 

9  2°  o 

3  8% 

SC  1 3 

16 

12.0% 

9.1% 

SCI  6 

17 

13.2% 

7  2% 

SC  leum 

12 

1 1.3% 

5  6% 

CPI 6  x  SPIcum 

15 

1 2.4% 

6  5% 

Composite 

1  1 

15.2% 

34% 

Raxleigh  (Lin  Regi 

24 

120  9% 

23  4% 

Rax  lei  gh  Cl-G  Reg) 

14 

132.1% 

0  0% 

Raxleigh  MMAL 

N  A 

0.0% 

0.0% 

Beta 

27 

627.0% 

870  2% 

TY  Results  for  Mixed  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPE 

17 

13  6% 

7.9% 

CPI  6 

16 

14  2% 

7.5% 

CPlcum 

14 

14  4% 

5.0% 

SPI3 

7 

1 3.5% 

3  6% 

SPI6 

9 

14.3% 

3.5% 

SPIcum 

11 

15  1% 

1.9% 

SCI  3 

15 

12  2°  o 

8.5% 

SC  16 

16 

13  6% 

7  8% 

SCicum 

12 

14  4% 

3  8% 

CPI 6  \  SPIcum 

15 

14.3% 

64% 

Composite 

14 

14.5% 

4  4% 

Rax  lei gh  (Lin  Reg) 

10 

10.8% 

7.0% 

Rax  lei gh  (  T-G  Regj 

25 

44.3% 

31  3% 

Raxleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

10 

13  1% 

6  1% 

Best  Performing  Method(s) 

Overall  SPI6  SPIcum 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPIcum 

CP16  x  SPIcum 

SPI3 

SCI6 


TY  Results  for  Mixed  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

15 

16  7% 

3  3% 

CP16 

1  1 

16.3% 

1.1% 

CPlcum 

13 

17.6% 

0.9% 

SPI3 

12 

15  1% 

3.0% 

SP16 

9 

14.3% 

2.6% 

SPIcum 

6 

14  1% 

0.9% 

SC  1 3 

13 

14  8% 

4.3% 

SC  16 

7 

13  5% 

2.6% 

SCicum 

6 

14.8% 

0.4% 

CPI6  x  SPIcum 

5 

13.3% 

11% 

Composite 

1  1 

17.0% 

0.6% 

Ravleigh  (Lin  Reg) 

24 

1070.0% 

1443.0% 

Rayleigh  (T-G  Reg) 

184.2% 

1503% 

Ravleigh  MMAE 

N  A 

0.0% 

0.0% 

Beta 

20 

23  9% 

12.3% 

TY  Results  for  Mixed  Data  >75%  to  100% 
Complete 

Score 

MAPI: 

SDAPE 

CPI3 

16 

8  3% 

2.8% 

CP16 

12 

8.2% 

1 .9% 

CPlcum 

12 

9.0% 

0.8% 

SPI3 

1 1 

9.7% 

0.5% 

SP16 

1  I 

9  8% 

0.4% 

SPIcum 

13 

10.7% 

0.5% 

SCI3 

1  1 

7.6% 

2.1% 

SC16 

9 

7  8% 

15% 

SCicum 

1  1 

9.4% 

0.6% 

CP16  x  SPIcum 

13 

8  5% 

1  6% 

Composite 

12 

9.4% 

0.7% 

Rayleigh  (Lm  Reg) 

24 

21.3% 

5.6% 

Rayleigh  (T-G  Reg) 

26 

38.3% 

49.3% 

Ravleigh  MMAL: 

N  A 

0  0% 

0.0% 

Beta 

1  1 

4.9% 

2.3% 
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|  CY95  Results  for  All  Mixed  Data  f 

Score 

MAPE 

SDAPE 

CPI  3 

14 

1 1  6% 

7.3% 

CPI6 

14 

11.8% 

7  1% 

CPIcum 

13 

12  1% 

62% 

SP13 

4 

9.8% 

5.8% 

SP16 

5 

10.0% 

5  8% 

SPIcum 

5 

10.1% 

5  5% 

SC  13 

10 

9.7% 

7  7% 

SCI  6 

1  1 

10  1% 

7.2% 

SCicum 

7 

10  4% 

5  8% 

CP16  x  SPIcum 

9 

10.1% 

6.6% 

Composite 

10 

11.7% 

6.0% 

Rayleigh  (Ltn  Reg) 

26 

761  1% 

2286  9% 

Rayleigh  (T-G  Reg) 

20 

73.1% 

76  5% 

Rayleigh  MMAE 

24 

141  5% 

369  7% 

Beta 

1 25  5% 

337  8% 

CY95  Results  for  Mixed  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CP  13 

17 

12.9% 

5.2% 

CP16 

19 

13  4% 

5  6% 

CP]  cum 

13 

12.3% 

4.8% 

SPI3 

4 

5.7% 

4  1% 

SP16 

8 

6.4% 

4.9% 

SPIcum 

4 

5.5% 

4.4% 

SC  13 

17 

8.4% 

8.2% 

SCI6 

18 

9.6% 

7.4% 

SCIcum 

13 

7.6% 

5  8% 

CPI 6  \  SPIcum 

16 

88% 

6.7% 

Composite 

14 

10.9% 

5.0% 

Ravleigh  (Lin  Reg! 

25 

127.0% 

31.5% 

Rayleigh  iT-G  Reg) 

14 

140.9% 

0.0% 

Ravleigh  MMAE 

30 

731  0% 

811.7% 

Beta 

28 

504.5% 

635  7% 

CY  95  Results  for  Mixed  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

17 

13.4% 

9.5% 

CP16 

16 

13.9% 

9.2% 

CPIcum 

16 

34.1% 

6.9% 

SPI3 

Q 

13.4% 

5.6% 

SPI6 

1  1 

14  1% 

5  5% 

SPIcum 

13 

14  8% 

4  1% 

SC  13 

17 

12.2% 

9.9% 

SC  16 

16 

3  3.4% 

9  4% 

SCicum 

14 

14.2% 

5  7% 

CP16  x  SPIcum 

16 

14  0% 

8  1% 

Composite 

17 

14  3% 

64% 

Rayleigh  ( Lin  Reg ) 

") 

7.3% 

15% 

Ravleigh  (T-G  Reg) 

26 

38.0% 

380% 

Ra\ leigh  MMAE 

8 

8.2% 

6.0% 

Beta 

14 

10  0% 

9.4% 

Best  Performing  Method(s) 


Overall  SPI3 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


SPI3  SPIcum 
CP16  x  SPIcum 
Rayleigh  (Lin  Reg) 
SC16 


CY  95  Results  for  Mixi 

Com 

ed  Data  >25%  to  50% 
plete 

Score 

MAPE 

SDAPE 

CPI3 

39 

15  9% 

4  9% 

CPI6 

!  1 

15.5% 

2.3% 

CPIcum 

12 

16.9% 

15% 

SP13 

14 

3  4.3% 

4.2% 

SP36 

12 

13.5% 

4.3% 

SPIcum 

8 

13.4% 

2.7% 

SCI3 

16 

14.0% 

6  1% 

SC16 

1 1 

12.7% 

4.5% 

SCicum 

10 

14  1% 

to 

-U 

CP16  x  SPIcum 

7 

12.6% 

3.2% 

Composite 

12 

162% 

1.6% 

Ravleigh  (Lin  Reg) 

30 

2932  6% 

4395.6% 

Rayleigh  (T-G  Reg) 

28 

135.3% 

108  3% 

Rayleigh  MMAE 

26 

45.7% 

64.0% 

Beta 

24 

21  6% 

19.2% 

CV95  Results  for  Mixed  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

13 

3  6% 

2.2% 

CP16 

11 

3.6% 

1.5% 

CPIcum 

10 

4.2% 

0.7% 

SPI3 

9 

4.7% 

0.2% 

SPI6 

11 

4  8% 

0.3% 

SPIcum 

13 

5.3% 

0.5% 

SCI3 

10 

3.2% 

1.8% 

SCI6 

8 

3.3% 

1.3% 

SCicum 

1  I 

4.5% 

0.6% 

CPI6  x  SPIcum 

1  1 

3  8% 

1.4% 

Composite 

10 

4.4% 

0.6% 

Rayleigh  (Lin  Reg) 

23 

17.4% 

5.3% 

Rayleigh  (T-G  Reg) 

27 

35.1% 

51.1% 

Ravleigh  MMAE 

25 

22  0% 

10.1% 

Beta 

18 

4.5% 

2.3% 
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TV  Results  for  All  Army  Data  | 

Score 

MAPE 

SDAPE 

CPI  3 

9 

12  4% 

12  8% 

CPI6 

4 

1 1 .0% 

11.6% 

CPIcum 

6 

1 1  4% 

11.9% 

SP13 

19 

14  8% 

20.7% 

SP16 

9 

12  0% 

14  0% 

SPIcum 

14 

12  7% 

19  8% 

SC  13 

17  8% 

23.8% 

SCI  6 

15 

13.1% 

16  2% 

SCI  cum 

19 

13.6% 

21.2% 

CPI6  \  SPicum 

13 

13  0% 

16.1% 

Composite 

1 0  8% 

10.9% 

Rayleigh  (  Lin  Reg  1 

24 

147  8% 

359.6% 

Ravi ei  eh  (T-G  Reg  > 

26 

193.5% 

587  9% 

Rayleigh  MMAE 

N'A 

0  0% 

0  0% 

Beta 

28 

1055.5% 

6544.0% 

TV  Results  for  Army  Data  0%  to  25%  Complete 

Score 

MAPE 

SDAPE 

CPI  3 

8 

22  2% 

16  4% 

CPI6 

5 

21  1% 

15  7% 

CPIcum 

5 

21.7% 

15  1% 

SP13 

17 

26  9°  o 

32.0% 

SP16 

10 

22.5% 

20.1% 

SPicum 

15 

23.9% 

32  8% 

SC  13  ’ 

■>"> 

32. 1  % 

34  6% 

SCi6 

14 

25  8% 

23  8% 

SCtcum 

2ij 

27.5% 

34  3% 

CP  16  \  SPicum 

14 

24  8% 

25  2% 

Composite 

19.5% 

13  7% 

Rayleigh  (l.in  Reg) 

24  i 

1  1 1  3% 

46  4% 

Ravieigh  -G  Reg ) 

26 

140  7% 

187  1% 

Ravieigh  MMAE 

N  A 

0  0% 

0  0% 

Beta 

28 

2638  6% 

10306.1% 

TV  Results  for  Army  Data  >50%  to  75% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

1  n 

9.2% 

6.8% 

CPI  6 

8  0% 

5.3% 

CPIcum 

5 

8  1% 

5.6% 

SP13 

18 

10  8% 

8.6% 

SPJ6 

14 

9  6°  o 

7.6% 

SPicum 

9 

9.0% 

6.0% 

SC  13 

21 

12.5% 

12.0% 

SC  16 

18 

9.7% 

9.1% 

SCI  cum 

9 

8  7% 

6  6% 

CPI 6  \  SPicum 

15 

9.6% 

7.7% 

Composite 

5 

8  2% 

5  5% 

Ra\ lt’igh  ( Lin  Reg} 

25 

190.0% 

416  7% 

Ravleigh  t T-G  Reg) 

27 

245  0% 

578  2% 

Ra\  icigh  MM  At. 

N  A 

0  0% 

0.0% 

Beta 

^5 

52  7% 

613% 

Best  Performing  Method(s) 

Overall  Composite 


0%  -  25% 
>25%  -  50% 
>50%  -  75% 
>75%-  100% 


Composite 

CPI6 

CPI6 

CPIcum  SP16  Composite 


TV  Results  for  Army  Data  >25%  to  50% 
Complete 

Score 

MAPE 

SDAPE 

CPI3 

7 

13.1% 

10.8% 

CP16 

T 

12  8% 

10.0% 

CPIcum 

5 

12  9% 

1 0.4% 

SPI3 

15 

17  6% 

12.7% 

SPI6 

12 

17.1% 

12.2% 

SPicum 

8 

15  4% 

10.4% 

SCI3 

18 

20.3% 

15.3% 

SC  16 

15 

17  7% 

12  4% 

SClcum 

9 

15  2% 

12.2% 

CP16x  SPicum 

12 

16  1% 

12.4% 

Composite 

3 

12  8% 

10.2% 

Rayleigh  (Lin  Reg) 

'»'■> 

2216% 

598  7% 

Rayleigh  (T-G  Reg) 

20  ! 

159.1% 

301  7% 

Rayleigh  MMAL 

N'A 

0  0% 

0  0% 

Beta 

24 

1783  4% 

8422.2% 

TV  Results  for  Army  Data  >75%  to  100% 
Complete 

Score 

MAPE 

SDAPE 

CP13 

14 

5.9% 

6.1% 

CP16 

12 

5.4% 

5.8% 

CPIcum 

3 

4.6% 

3.9% 

SPI3 

10 

5.0% 

5.7% 

SPI6 

3 

4.3% 

4.2% 

SPicum 

7 

4.9% 

4.3% 

SCI3 

18 

66% 

10.8% 

SC  16 

7 

4.7% 

5.5% 

SClcum 

7 

5  0% 

4  2% 

CPI6  x  SPicum 

16 

6  0% 

62% 

Composite 

3 

4  6% 

3  9% 

Rayleigh  (Lin  Reg) 

22 

110  0% 

249  2% 

Rayleigh  (T-G  Reg) 

24 

205.2% 

776.5% 

Rayleigh  MMAE 

N'A 

0.0% 

0.0% 

Beta 

20 

1 1  9% 

25  0% 

172 


Results  for  Army  Contracts  Using  Inflation  Adjusted  Data 


j  CY95  Results  for  All  Army  Data  j 

Score 

MAPE 

SDAPE 

CPI  5 

8 

118% 

1 1  9% 

CPI  6 

4 

10  8% 

10.7% 

CPI  cum 

6 

111% 

11.1% 

SPI3 

19 

15.5% 

24.0% 

SPI6 

10 

12.7% 

15  4% 

SPlcum 

14 

13.5% 

23.1% 

SCI? 

22 

18  2% 

76.85b 

SC  16 

14 

13  8% 

18  0% 

SCI  cum 

!  9 

14.5% 

24  8% 

CPI6  \  SPlcum 

14 

13  7% 

18  1% 

Composite 

2 

10  6% 

10.0% 

Rayleigh  l  Lin  Reg} 

24 

145.2% 

445  8% 

Ravleigh  iT-G  Reg) 

28 

514.5% 

4307.0% 

Ravleigh  MMAF 

26 

326.4% 

674.0% 

Beta 

30 

2143  4% 

15882  0% 

CY95  Results  for  Army  Data  0%  to  25% 
Complete 

Score 

MAPE 

SDAPE 

CPI  3 

8 

18.9% 

16.8% 

CP16 

5 

1 7  8°  o 

16  0% 

CPIcum 

5 

18  3% 

15  6% 

SP13 

17 

26  9°  o 

39  6% 

SPI6 

10 

22.3% 

24  6% 

SPlcum 

15 

^4  ~*% 

40  1% 

SCI? 

22 

33  1% 

42.2% 

SC"  IP 

14 

26  1% 

29  1% 

SCieuni 

20 

28.2% 

42  1% 

CPR  x  SPlcum 

14 

25  0% 

30.8% 

Composite 

-s 

16  4% 

13  7% 

Ravleigh  1 1. in  Reg) 

1 1 1.0% 

55.9% 

Rav  leigh  >  T-G  Reg} 

25 

1 12.0% 

45. 1% 

Ravleigh  MMAF 

28 

618  9% 

986  6% 

Beta 

30 

5873.7% 

28227  0% 
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